2024-03-16T17:43:41.553 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-16T17:43:41.553 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi --owner scheduled_leonidus@teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/acls} --archive /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024 -- /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024/orig.config.yaml 2024-03-16T17:43:41.598 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T17:43:41.679 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024 branch: wip-lusov-qdb-squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/acls} email: null first_in_suite: false job_id: '7602024' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb-squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Reduced data availability - Degraded data redundancy sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b kclient: syntax: v1 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb-squid sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b owner: scheduled_leonidus@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6918 sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b sleep_before_teardown: 0 subset: 111/10000 suite: fs suite_branch: wip-lusov-qdb-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMWDTjOWyp/xznfLFtNdGdjRytqkO/3ebZ6djnkvUAL2aYk9ZxFLqqX4KHBa7qOMy3OriVHGpGWzJ619MA+ZWK4= smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMHGJiu0M9WpvHJtGRtV6+Bjf+tQAbbC+q4LpxhEK8u3bCpPcuXOnJC7Kux/5hu6k9cb6uWa/Jm96mHFtJ48DbI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_acls teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-15_11:44:52 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-16T17:43:41.679 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa; will attempt to use it 2024-03-16T17:43:41.680 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks 2024-03-16T17:43:41.680 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-16T17:43:41.682 INFO:teuthology.task.internal:Checking packages... 2024-03-16T17:43:41.707 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b' 2024-03-16T17:43:41.707 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T17:43:41.707 INFO:teuthology.packaging:ref: None 2024-03-16T17:43:41.707 INFO:teuthology.packaging:tag: None 2024-03-16T17:43:41.707 INFO:teuthology.packaging:branch: wip-lusov-qdb-squid 2024-03-16T17:43:41.707 INFO:teuthology.packaging:sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-16T17:43:41.707 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb-squid 2024-03-16T17:43:41.843 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1681.gaba3c1d2 2024-03-16T17:43:41.844 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-16T17:43:41.854 INFO:teuthology.task.internal:no buildpackages task found 2024-03-16T17:43:41.854 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-16T17:43:41.875 INFO:teuthology.task.internal:Saving configuration 2024-03-16T17:43:41.892 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-16T17:43:41.900 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T17:43:41.928 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 17:38:05.455455', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMWDTjOWyp/xznfLFtNdGdjRytqkO/3ebZ6djnkvUAL2aYk9ZxFLqqX4KHBa7qOMy3OriVHGpGWzJ619MA+ZWK4='} 2024-03-16T17:43:41.952 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 17:38:05.454244', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMHGJiu0M9WpvHJtGRtV6+Bjf+tQAbbC+q4LpxhEK8u3bCpPcuXOnJC7Kux/5hu6k9cb6uWa/Jm96mHFtJ48DbI='} 2024-03-16T17:43:41.952 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-16T17:43:41.961 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-16T17:43:41.961 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-16T17:43:41.961 INFO:teuthology.run_tasks:Running task console_log... 2024-03-16T17:43:42.042 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6a5aa310d0>, signals=[15]) 2024-03-16T17:43:42.042 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-16T17:43:42.049 INFO:teuthology.task.internal:Opening connections... 2024-03-16T17:43:42.049 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T17:43:42.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:43:42.145 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T17:43:42.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:43:42.232 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-16T17:43:42.241 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-03-16T17:43:42.258 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-03-16T17:43:42.259 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-03-16T17:43:42.316 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="CentOS Stream" 2024-03-16T17:43:42.316 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="9" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:ID="centos" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="rhel fedora" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="9" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el9" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:LOGO="fedora-logo-icon" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://centos.org/" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T17:43:42.317 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T17:43:42.318 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-03-16T17:43:42.344 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-03-16T17:43:42.362 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-03-16T17:43:42.362 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-03-16T17:43:42.419 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-03-16T17:43:42.419 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-03-16T17:43:42.419 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-03-16T17:43:42.419 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-03-16T17:43:42.419 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-03-16T17:43:42.419 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-03-16T17:43:42.419 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-16T17:43:42.419 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-03-16T17:43:42.420 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-03-16T17:43:42.420 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-16T17:43:42.420 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-03-16T17:43:42.420 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-16T17:43:42.420 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-16T17:43:42.420 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-16T17:43:42.420 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-03-16T17:43:42.444 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-16T17:43:42.454 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-16T17:43:42.465 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-16T17:43:42.465 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-03-16T17:43:42.468 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-03-16T17:43:42.482 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-16T17:43:42.491 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-16T17:43:42.491 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-03-16T17:43:42.524 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-03-16T17:43:42.623 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-16T17:43:42.682 INFO:teuthology.run_tasks:Running task kernel... 2024-03-16T17:43:42.705 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-16T17:43:42.705 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-03-16T17:43:42.706 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-03-16T17:43:42.706 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T17:43:42.707 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T17:43:42.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T17:43:42.723 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-16T17:43:42.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T17:43:42.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-16T17:43:42.927 DEBUG:teuthology.task.kernel:sha1 for client.0 is 3845b7ac715a3ff198582638d199a90e30e254da 2024-03-16T17:43:42.927 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "3845b7ac715a3ff198582638d199a90e30e254da"... 2024-03-16T17:43:42.927 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-03-16T17:43:42.944 INFO:teuthology.orchestra.run.smithi082.stdout:5.14.0-361.el9.x86_64 2024-03-16T17:43:42.944 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3845b7ac715a3ff198582638d199a90e30e254da 2024-03-16T17:43:42.945 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-16T17:43:42.945 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum upgrade -y linux-firmware 2024-03-16T17:43:43.108 DEBUG:teuthology.task.kernel:sha1 for client.1 is 3845b7ac715a3ff198582638d199a90e30e254da 2024-03-16T17:43:43.108 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "3845b7ac715a3ff198582638d199a90e30e254da"... 2024-03-16T17:43:43.108 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-03-16T17:43:43.125 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-361.el9.x86_64 2024-03-16T17:43:43.125 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 3845b7ac715a3ff198582638d199a90e30e254da 2024-03-16T17:43:43.125 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-16T17:43:43.126 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum upgrade -y linux-firmware 2024-03-16T17:43:45.068 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-16T17:43:45.685 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-16T17:43:46.719 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2024-03-16T17:43:47.077 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-03-16T17:43:47.534 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-16T17:43:49.546 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-03-16T17:43:49.851 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 3.3 MB/s | 8.0 MB 00:02 2024-03-16T17:43:51.744 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-03-16T17:43:55.758 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-16T17:43:57.114 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 15 kB 00:00 2024-03-16T17:43:58.141 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-03-16T17:44:02.001 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 717 kB/s | 6.1 MB 00:08 2024-03-16T17:44:03.075 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-16T17:44:03.343 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 15 kB 00:00 2024-03-16T17:44:05.127 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-16T17:44:05.411 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-16T17:44:05.417 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-16T17:44:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-16T17:44:05.417 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-16T17:44:05.417 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-03-16T17:44:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-16T17:44:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:44:05.417 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-16T17:44:05.417 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-03-16T17:44:05.418 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 1 Package 2024-03-16T17:44:05.418 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:44:05.418 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 375 M 2024-03-16T17:44:05.418 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-16T17:44:10.002 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-03-16T17:44:12.285 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-03-16T17:44:12.292 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 1 Package 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 375 M 2024-03-16T17:44:12.293 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-03-16T17:44:13.943 INFO:teuthology.orchestra.run.smithi090.stdout:linux-firmware-20240219-143.el9.noarch.rpm 44 MB/s | 375 MB 00:08 2024-03-16T17:44:13.944 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-16T17:44:13.945 INFO:teuthology.orchestra.run.smithi090.stdout:Total 44 MB/s | 375 MB 00:08 2024-03-16T17:44:13.945 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-16T17:44:14.067 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-16T17:44:14.067 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-16T17:44:16.473 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-16T17:44:16.474 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-16T17:44:20.893 INFO:teuthology.orchestra.run.smithi082.stdout:linux-firmware-20240219-143.el9.noarch.rpm 44 MB/s | 375 MB 00:08 2024-03-16T17:44:20.895 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-03-16T17:44:20.895 INFO:teuthology.orchestra.run.smithi082.stdout:Total 44 MB/s | 375 MB 00:08 2024-03-16T17:44:20.895 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-03-16T17:44:20.988 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-03-16T17:44:20.988 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-03-16T17:44:23.462 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-03-16T17:44:23.462 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-03-16T17:44:23.675 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-16T17:44:23.883 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-16T17:44:23.975 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-16T17:44:25.788 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-16T17:44:25.789 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-16T17:44:26.187 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-16T17:44:26.187 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:44:26.187 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-03-16T17:44:26.187 INFO:teuthology.orchestra.run.smithi090.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-16T17:44:26.187 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:44:26.187 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-16T17:44:26.545 INFO:teuthology.task.kernel:Downloading kernel 3845b7ac715a3ff198582638d199a90e30e254da on client.1... 2024-03-16T17:44:26.546 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3845b7ac715a3ff198582638d199a90e30e254da 2024-03-16T17:44:26.691 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/ 2024-03-16T17:44:26.691 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_g3845b7ac715a-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-16T17:44:26.694 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi090.front.sepia.ceph.com to complete... 2024-03-16T17:44:29.672 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-03-16T17:44:29.919 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-16T17:44:30.009 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-16T17:44:31.299 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-16T17:44:31.300 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-16T17:44:31.694 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-16T17:44:31.695 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:44:31.695 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-03-16T17:44:31.695 INFO:teuthology.orchestra.run.smithi082.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-16T17:44:31.695 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:44:31.695 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-03-16T17:44:32.032 INFO:teuthology.task.kernel:Downloading kernel 3845b7ac715a3ff198582638d199a90e30e254da on client.0... 2024-03-16T17:44:32.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3845b7ac715a3ff198582638d199a90e30e254da 2024-03-16T17:44:32.173 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/ 2024-03-16T17:44:32.174 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_g3845b7ac715a-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-16T17:44:32.176 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi082.front.sepia.ceph.com to complete... 2024-03-16T17:44:47.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16 17:44:47 URL:https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_g3845b7ac715a-1.x86_64.rpm [336075563/336075563] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-16T17:44:47.119 INFO:teuthology.orchestra.run.smithi090.stderr:FINISHED --2024-03-16 17:44:47-- 2024-03-16T17:44:47.119 INFO:teuthology.orchestra.run.smithi090.stderr:Total wall clock time: 20s 2024-03-16T17:44:47.119 INFO:teuthology.orchestra.run.smithi090.stderr:Downloaded: 1 files, 321M in 20s (16.1 MB/s) 2024-03-16T17:44:47.121 INFO:teuthology.task.kernel:Installing kernel 3845b7ac715a3ff198582638d199a90e30e254da on client.1... 2024-03-16T17:44:47.121 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-16T17:44:49.044 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying... ######################################## 2024-03-16T17:44:49.114 INFO:teuthology.orchestra.run.smithi090.stdout:Preparing... ######################################## 2024-03-16T17:44:49.193 INFO:teuthology.orchestra.run.smithi090.stdout:Updating / installing... 2024-03-16T17:44:56.709 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-6.8.0_rc7_g3845b7ac715a-1 ######################################## 2024-03-16T17:45:02.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16 17:45:02 URL:https://2.chacra.ceph.com/r/kernel/testing/3845b7ac715a3ff198582638d199a90e30e254da/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_g3845b7ac715a-1.x86_64.rpm [336075563/336075563] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-16T17:45:02.618 INFO:teuthology.orchestra.run.smithi082.stderr:FINISHED --2024-03-16 17:45:02-- 2024-03-16T17:45:02.618 INFO:teuthology.orchestra.run.smithi082.stderr:Total wall clock time: 30s 2024-03-16T17:45:02.618 INFO:teuthology.orchestra.run.smithi082.stderr:Downloaded: 1 files, 321M in 30s (10.7 MB/s) 2024-03-16T17:45:02.621 INFO:teuthology.task.kernel:Installing kernel 3845b7ac715a3ff198582638d199a90e30e254da on client.0... 2024-03-16T17:45:02.621 DEBUG:teuthology.orchestra.run.smithi082:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-16T17:45:04.545 INFO:teuthology.orchestra.run.smithi082.stdout:Verifying... ######################################## 2024-03-16T17:45:04.665 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing... ######################################## 2024-03-16T17:45:04.745 INFO:teuthology.orchestra.run.smithi082.stdout:Updating / installing... 2024-03-16T17:45:12.285 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-6.8.0_rc7_g3845b7ac715a-1 ######################################## 2024-03-16T17:45:17.389 INFO:teuthology.orchestra.run.smithi090.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-16T17:45:26.362 INFO:teuthology.orchestra.run.smithi082.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-16T17:45:40.716 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi090.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-16T17:45:40.716 DEBUG:teuthology.orchestra.run.smithi090:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-16T17:45:40.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:40.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/System.map 2024-03-16T17:45:40.799 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/config 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-16T17:45:40.800 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel 2024-03-16T17:45:40.801 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu/mce 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm.ko 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/ansi_cprng.ko 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/anubis.ko 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/arc4.ko 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_memcpy.ko 2024-03-16T17:45:40.802 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_pq.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_tx.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_xor.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/raid6test.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/authenc.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/authencesn.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blake2b_generic.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blowfish_common.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blowfish_generic.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/camellia_generic.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast5_generic.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast6_generic.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast_common.ko 2024-03-16T17:45:40.803 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/ccm.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cmac.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crc32_generic.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crypto_engine.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crypto_user.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cts.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/des_generic.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/dh_generic.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/echainiv.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/essiv.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/fcrypt.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/khazad.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/lz4.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/lz4hc.ko 2024-03-16T17:45:40.804 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/md4.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/michael_mic.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/pcbc.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/pcrypt.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/rmd160.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/seed.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/serpent_generic.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/tcrypt.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/tea.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/twofish_common.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/twofish_generic.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/vmac.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/wp512.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xcbc.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xor.ko 2024-03-16T17:45:40.805 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xxhash_generic.ko 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/acpi_pad.ko 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/custom_method.ko 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/ec_sys.ko 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/platform_profile.ko 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/sbs.ko 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/sbshc.ko 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/video.ko 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/acard-ahci.ko 2024-03-16T17:45:40.806 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/ahci_platform.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/ata_generic.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/libahci_platform.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_acpi.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ali.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_amd.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_artop.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_atiixp.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_atp867x.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cmd640.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cmd64x.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cypress.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_efar.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt366.ko 2024-03-16T17:45:40.807 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt37x.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_it8213.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_it821x.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_jmicron.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_marvell.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_mpiix.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_netcell.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ninja32.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ns87410.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ns87415.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_oldpiix.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_opti.ko 2024-03-16T17:45:40.808 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_optidma.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pcmcia.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_piccolo.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_rdc.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sch.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_serverworks.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sil680.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sis.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sl82c105.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_triflex.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_via.ko 2024-03-16T17:45:40.809 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pdc_adma.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_inic162x.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_mv.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_nv.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_promise.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_qstor.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sil.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sil24.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sis.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_svw.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sx4.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_uli.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_via.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_vsc.ko 2024-03-16T17:45:40.810 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/auxdisplay 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/auxdisplay/ks0108.ko 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/bcma 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/bcma/bcma.ko 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/aoe 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/aoe/aoe.ko 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/brd.ko 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/drbd 2024-03-16T17:45:40.811 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/drbd/drbd.ko 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/floppy.ko 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/loop.ko 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/mtip32xx 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/nbd.ko 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/null_blk 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/null_blk/null_blk.ko 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/pktcdvd.ko 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/rbd.ko 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/virtio_blk.ko 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkback 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-16T17:45:40.812 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkfront.ko 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/zram 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/zram/zram.ko 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hangcheck-timer.ko 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/via-rng.ko 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-16T17:45:40.813 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/lp.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/mwave 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/mwave/mwave.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ppdev.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tlclk.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-16T17:45:40.814 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/uv_mmtimer.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/virtio_console.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-16T17:45:40.815 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp/ccp.ko 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_common 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/padlock-aes.ko 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/padlock-sha.ko 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/virtio 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dca 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dca/dca.ko 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/devfreq 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-16T17:45:40.816 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/dw 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/ioat 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/amd64_edac.ko 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/e752x_edac.ko 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/edac_mce_amd.ko 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i3000_edac.ko 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i3200_edac.ko 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i5100_edac.ko 2024-03-16T17:45:40.817 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i5400_edac.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i7300_edac.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i7core_edac.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i82975x_edac.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/ie31200_edac.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/sb_edac.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/x38_edac.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware/edd.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio/gpio-ich.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-16T17:45:40.818 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-appleir.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-aureal.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-axff.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-dr.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-elecom.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-elo.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-emsff.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gaff.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gt683r.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gyration.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtekff.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-hyperv.ko 2024-03-16T17:45:40.819 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-icade.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-keytouch.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-kye.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-lcpower.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-led.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-lenovo.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-multitouch.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-ortek.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-penmount.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-petalynx.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-picolcd.ko 2024-03-16T17:45:40.820 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-pl.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-plantronics.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-primax.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-rmi.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-common.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-16T17:45:40.821 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-saitek.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-samsung.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sjoy.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sony.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-speedlink.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-steelseries.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sunplus.ko 2024-03-16T17:45:40.822 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-tivo.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-tmff.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-topseed.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-twinhan.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-uclogic.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-waltop.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-wiimote.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-xinmo.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-zpff.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-zydacron.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/uhid.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/wacom.ko 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv 2024-03-16T17:45:40.823 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_balloon.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_utils.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_vmbus.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/abituguru.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/abituguru3.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ad7414.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ad7418.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adc128d818.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1025.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1026.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1029.ko 2024-03-16T17:45:40.824 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1031.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm9240.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ads7828.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7410.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7411.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7462.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7470.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7475.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7x10.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/amc6821.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/applesmc.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asb100.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asc7621.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-16T17:45:40.825 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/atxp1.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/coretemp.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/dme1737.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ds1621.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ds620.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/emc1403.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/emc6w201.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f71805f.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f71882fg.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f75375s.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/fam15h_power.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/fschmd.ko 2024-03-16T17:45:40.826 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/g760a.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/g762.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/gl518sm.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/gl520sm.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/i5500_temp.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/i5k_amb.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ibmaem.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ibmpex.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ina209.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ina2xx.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/it87.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/k10temp.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/k8temp.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lineage-pem.ko 2024-03-16T17:45:40.827 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm63.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm73.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm75.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm77.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm78.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm80.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm83.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm85.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm87.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm90.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm92.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm93.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95234.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95241.ko 2024-03-16T17:45:40.828 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95245.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc2945.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4151.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4215.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4222.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4245.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4260.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4261.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max16065.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max1619.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max1668.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max197.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6639.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6650.ko 2024-03-16T17:45:40.829 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6697.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/mcp3021.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6683.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6775-core.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6775.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct7802.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pc87360.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pc87427.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pcf8591.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-16T17:45:40.830 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/powr1220.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch5627.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch5636.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sht15.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sht21.ko 2024-03-16T17:45:40.831 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/shtc1.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sis5595.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47b397.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47m1.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47m192.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/thmc50.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp102.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp103.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp401.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp421.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/via-cputemp.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/via686a.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/vt1211.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/vt8231.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83627ehf.ko 2024-03-16T17:45:40.832 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83627hf.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83781d.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83791d.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83792d.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83793.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83795.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83l785ts.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83l786ng.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-16T17:45:40.833 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-16T17:45:40.834 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-dev.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-mux.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-smbus.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-stub.ko 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband 2024-03-16T17:45:40.835 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_core.ko 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/cxgb4 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx4 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx5 2024-03-16T17:45:40.836 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mthca 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/ipoib 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/iser 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/isert 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srp 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srpt 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input 2024-03-16T17:45:40.837 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/gameport.ko 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/joydev.ko 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/keyboard 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/appletouch.ko 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/bcm5974.ko 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/cyapatp.ko 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/sermouse.ko 2024-03-16T17:45:40.838 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/rmi4 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/altera_ps2.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/arc_ps2.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/serio_raw.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/sparse-keymap.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-blinkm.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-lm3530.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-lp3944.ko 2024-03-16T17:45:40.839 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-ss4200.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/bcache 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/bcache/bcache.ko 2024-03-16T17:45:40.840 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-bio-prison.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-cache-smq.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-cache.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-crypt.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-delay.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-flakey.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-log-userspace.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-multipath.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-queue-length.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-raid.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-round-robin.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-service-time.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-switch.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-thin-pool.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-verity.ko 2024-03-16T17:45:40.841 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/persistent-data 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid0.ko 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid1.ko 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid10.ko 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid456.ko 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptbase.ko 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptctl.ko 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptfc.ko 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptlan.ko 2024-03-16T17:45:40.842 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptsas.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptscsih.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptspi.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/lpc_ich.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/lpc_sch.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/mfd-core.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/sm501.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/viperboard.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/vx855.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/wl1273-core.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/altera-stapl 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-16T17:45:40.843 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/apds9802als.ko 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/apds990x.ko 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/bh1770glc.ko 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/cb710 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/cb710/cb710.ko 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/at24.ko 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/max6875.ko 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/enclosure.ko 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/hpilo.ko 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/isl29003.ko 2024-03-16T17:45:40.844 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/isl29020.ko 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei-me.ko 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei-txe.ko 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei.ko 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-gru 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-16T17:45:40.845 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tifm_7xx1.ko 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tifm_core.ko 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tsl2550.ko 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_balloon.ko 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_vmci 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/bonding 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/bonding/bonding.ko 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/b53 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-16T17:45:40.846 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6xxx 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dummy.ko 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/eql.ko 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-16T17:45:40.847 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/adaptec 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/agere 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/alteon 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/altera 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-16T17:45:40.848 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/alx 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-16T17:45:40.849 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/genet 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade/bna 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-16T17:45:40.850 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco/enic 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-16T17:45:40.851 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dnet.ko 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex/benet 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ethoc.ko 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/fealnx.ko 2024-03-16T17:45:40.852 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000e 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/fm10k 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/i40e 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/iavf 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-16T17:45:40.853 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igb 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igbvf 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/jme.ko 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-16T17:45:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/micrel 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi 2024-03-16T17:45:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/neterion 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/nvidia 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-16T17:45:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rdc 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rocker 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sfc 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-16T17:45:40.857 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/silan 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-16T17:45:40.858 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/tehuti 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ti 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet 2024-03-16T17:45:40.859 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/xircom 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/geneve.ko 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/hyperv 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ieee802154 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ifb.ko 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ipvlan 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/macvlan.ko 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/macvtap.ko 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio 2024-03-16T17:45:40.860 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mii.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/net_failover.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/netconsole.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/nlmon.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/amd.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-16T17:45:40.861 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm87xx.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/broadcom.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/cicada.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/davicom.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/dp83640.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/et1011c.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/icplus.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/lxt.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/marvell.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/micrel.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/national.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/phylink.ko 2024-03-16T17:45:40.862 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/qsemi.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/realtek.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/smsc.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/ste10Xp.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/vitesse.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_async.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pppoe.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pppox.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pptp.ko 2024-03-16T17:45:40.863 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip/slhc.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip/slip.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/sungem_phy.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/tap.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_random.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/tun.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/veth.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/virtio_net.ko 2024-03-16T17:45:40.864 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vmxnet3 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vxlan 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vxlan/vxlan.ko 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netback 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netfront.ko 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ntb 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ntb/ntb.ko 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host/nvme-core.ko 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host/nvme.ko 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport 2024-03-16T17:45:40.865 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport.ko 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_cs.ko 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_pc.ko 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_serial.ko 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/hotplug 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/pcie 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/xen-pcifront.ko 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/i82092.ko 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/pd6729.ko 2024-03-16T17:45:40.866 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl/intel 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/acerhdf.ko 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-16T17:45:40.867 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/hdaps.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel_ips.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-16T17:45:40.868 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/wmi.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps_parport.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/pps_core.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp/ptp.ko 2024-03-16T17:45:40.869 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp/ptp_kvm.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-em3027.ko 2024-03-16T17:45:40.870 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-max6900.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-16T17:45:40.871 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-x1205.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-9xxx.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-sas.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-xxxx.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/BusLogic.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/a100u2w.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aacraid 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/advansys.ko 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx 2024-03-16T17:45:40.872 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/am53c974.ko 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/arcmsr 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/atp870u.ko 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/be2iscsi 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bfa 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bfa/bfa.ko 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2fc 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2i 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ch.ko 2024-03-16T17:45:40.873 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/csiostor 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/dc395x.ko 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-16T17:45:40.874 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/dmx3191d.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esas2r 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esp_scsi.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fnic 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fnic/fnic.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hpsa.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hptiop.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hv_storvsc.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/initio.ko 2024-03-16T17:45:40.875 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ipr.ko 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ips.ko 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/isci 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/isci/isci.ko 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libfc 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libfc/libfc.ko 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libiscsi.ko 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libsas 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libsas/libsas.ko 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/lpfc 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-16T17:45:40.876 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid.ko 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mpt3sas 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvsas 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvumi.ko 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pm8001 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pmcraid.ko 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla1280.ko 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx 2024-03-16T17:45:40.877 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla4xxx 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/raid_class.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_debug.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ses.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/st.ko 2024-03-16T17:45:40.878 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/stex.ko 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/sym53c8xx_2 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/virtio_scsi.ko 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/wd719x.ko 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ssb 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ssb/ssb.ko 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/iscsi 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/loopback 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_file.ko 2024-03-16T17:45:40.879 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_iblock.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_mod.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_pscsi.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_user.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/tcm_fc 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-16T17:45:40.880 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/ipwireless 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-16T17:45:40.881 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/n_gsm.ko 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/n_hdlc.ko 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/nozomi.ko 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/8250 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/arc_uart.ko 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/jsm 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/core 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/host 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-16T17:45:40.882 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio.ko 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_aec.ko 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_cif.ko 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_pci_generic.ko 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_sercos3.ko 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/cdc-acm.ko 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/usblp.ko 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/usbtmc.ko 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/host 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image 2024-03-16T17:45:40.883 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image/mdc800.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image/microtek.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/adutux.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/appledisplay.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/emi26.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/emi62.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/ezusb.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/idmouse.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/iowarrior.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/ldusb.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/legousbtower.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/sisusbvga 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-16T17:45:40.884 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usb3503.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usblcd.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/uss720.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/yurex.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/phy 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/phy/phy-generic.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/aircable.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ark3116.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ch341.ko 2024-03-16T17:45:40.885 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cp210x.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cyberjack.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/empeg.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/io_ti.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ipaq.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ipw.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ir-usb.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/keyspan.ko 2024-03-16T17:45:40.886 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mct_u232.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mos7720.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mos7840.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/navman.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/omninet.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/opticon.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/option.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/oti6858.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/pl2303.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/qcaux.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/qcserial.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/quatech2.ko 2024-03-16T17:45:40.887 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/safe_serial.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/sierra.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ssu100.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/symbolserial.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb_debug.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/visor.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/whiteheat.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/uas.ko 2024-03-16T17:45:40.888 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-karma.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/usb-storage.ko 2024-03-16T17:45:40.889 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/vfio.ko 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost.ko 2024-03-16T17:45:40.890 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_net.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_scsi.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/apple_bl.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/lcd.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_balloon.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_mmio.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-16T17:45:40.891 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_ring.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters/ds2482.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters/ds2490.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-16T17:45:40.892 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/wire.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/hpwdt.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/i6300esb.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ib700wdt.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ibmasr.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-16T17:45:40.893 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/it87_wdt.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/machzwd.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/nv_tco.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/softdog.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/via_wdt.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-16T17:45:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/wdt_pci.ko 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/xen_wdt.ko 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-evtchn.ko 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-gntalloc.ko 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-gntdev.ko 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-pciback 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-privcmd.ko 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xenfs 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/9p 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/9p/9p.ko 2024-03-16T17:45:40.895 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/affs 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/affs/affs.ko 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/befs 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/befs/befs.ko 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/binfmt_misc.ko 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/btrfs 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/btrfs/btrfs.ko 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cachefiles 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cachefiles/cachefiles.ko 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ceph 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ceph/ceph.ko 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/coda 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/coda/coda.ko 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cramfs 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cramfs/cramfs.ko 2024-03-16T17:45:40.896 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/dlm 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/dlm/dlm.ko 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ecryptfs 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/f2fs 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/f2fs/f2fs.ko 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/fat.ko 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/msdos.ko 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/vfat.ko 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse/cuse.ko 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse/fuse.ko 2024-03-16T17:45:40.897 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/gfs2 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/gfs2/gfs2.ko 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfs 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfs/hfs.ko 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfsplus 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfsplus/hfsplus.ko 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/isofs 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/isofs/isofs.ko 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/jfs 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/jfs/jfs.ko 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/lockd 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/lockd/lockd.ko 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/minix 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/minix/minix.ko 2024-03-16T17:45:40.898 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/netfs 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/netfs/netfs.ko 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/blocklayout 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/filelayout 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/flexfilelayout 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfs.ko 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfsv3.ko 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfsv4.ko 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common/grace.ko 2024-03-16T17:45:40.899 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common/nfs_acl.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfsd 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfsd/nfsd.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nilfs2 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nilfs2/nilfs2.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-celtic.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-centeuro.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-croatian.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-cyrillic.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-gaelic.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-greek.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-iceland.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-inuit.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-roman.ko 2024-03-16T17:45:40.900 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-romanian.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-turkish.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1250.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1251.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1255.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp737.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp775.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp850.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp852.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp855.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp857.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp860.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp861.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp862.ko 2024-03-16T17:45:40.901 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp863.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp864.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp865.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp866.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp869.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp874.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp932.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp936.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp949.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp950.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_euc-jp.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-1.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-13.ko 2024-03-16T17:45:40.902 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-14.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-15.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-2.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-3.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-4.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-5.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-6.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-7.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-9.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-r.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-ru.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-u.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_utf8.ko 2024-03-16T17:45:40.903 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/cluster 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlm 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlmfs 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2.ko 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/overlayfs 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/overlayfs/overlay.ko 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/pstore 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/pstore/ramoops.ko 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/reiserfs 2024-03-16T17:45:40.904 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/reiserfs/reiserfs.ko 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/romfs 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/romfs/romfs.ko 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/client 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/client/cifs.ko 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common/cifs_arc4.ko 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common/cifs_md4.ko 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/squashfs 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/squashfs/squashfs.ko 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/sysv 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/sysv/sysv.ko 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/udf 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/udf/udf.ko 2024-03-16T17:45:40.905 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ufs 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ufs/ufs.ko 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/xfs 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/xfs/xfs.ko 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/rcu 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/rcu/rcutorture.ko 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/torture.ko 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/trace 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/asn1_encoder.ko 2024-03-16T17:45:40.906 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crc-itu-t.ko 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crc8.ko 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto/libarc4.ko 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto/libdes.ko 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/libcrc32c.ko 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lru_cache.ko 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4/lz4_compress.ko 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4/lz4hc_compress.ko 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/math 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/math/cordic.ko 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/raid6 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/raid6/raid6_pq.ko 2024-03-16T17:45:40.907 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/reed_solomon 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_bm.ko 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_fsm.ko 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_kmp.ko 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/zstd 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/zstd/zstd_compress.ko 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/mm 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/mm/hwpoison-inject.ko 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/6lowpan.ko 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_dest.ko 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_fragment.ko 2024-03-16T17:45:40.908 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_hop.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_mobility.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_routing.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_udp.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/garp.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/mrp.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/p8022.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/psnap.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/stp.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/8021q 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/8021q/8021q.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet.ko 2024-03-16T17:45:40.909 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_fd.ko 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_rdma.ko 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_virtio.ko 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/batman-adv 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/batman-adv/batman-adv.ko 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/br_netfilter.ko 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/bridge.ko 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-16T17:45:40.910 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-16T17:45:40.911 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtables.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ceph 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ceph/libceph.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core/failover.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core/pktgen.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp.ko 2024-03-16T17:45:40.912 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_diag.ko 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_ipv4.ko 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_ipv6.ko 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dns_resolver 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dns_resolver/dns_resolver.ko 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/dsa_core.ko 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_brcm.ko 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_dsa.ko 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_none.ko 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_trailer.ko 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/6lowpan 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-16T17:45:40.913 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/ieee802154.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ah4.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/esp4.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/fou.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/gre.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/inet_diag.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_gre.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_tunnel.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_vti.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ipcomp.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ipip.ko 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter 2024-03-16T17:45:40.914 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-16T17:45:40.915 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_bic.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_dctcp.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_diag.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_highspeed.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_htcp.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_hybla.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_illinois.ko 2024-03-16T17:45:40.916 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_lp.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_scalable.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_vegas.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_veno.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_westwood.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_yeah.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tunnel4.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/udp_diag.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/udp_tunnel.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ah6.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/esp6.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/fou6.ko 2024-03-16T17:45:40.917 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_tunnel.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_vti.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ipcomp6.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-16T17:45:40.918 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-16T17:45:40.919 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/sit.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/tunnel6.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/key 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/key/af_key.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_core.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_eth.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ip.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ip6.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_netlink.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ppp.ko 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/llc 2024-03-16T17:45:40.920 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/llc/llc.ko 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mac802154 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mac802154/mac802154.ko 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mpls 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mpls/mpls_gso.ko 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set.ko 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-16T17:45:40.921 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-16T17:45:40.922 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conncount.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-16T17:45:40.923 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_flow_table.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-16T17:45:40.924 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_log_syslog.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_irc.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_sip.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_tables.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_log.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_chain_nat.ko 2024-03-16T17:45:40.925 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_compat.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_ct.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_fib.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_hash.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_limit.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_log.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_masq.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_nat.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_numgen.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_queue.ko 2024-03-16T17:45:40.926 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_quota.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_redir.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_reject.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_reject_inet.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_socket.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_tunnel.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_AUDIT.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CT.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_DSCP.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_HL.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_HMARK.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-16T17:45:40.927 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_LED.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_LOG.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NETMAP.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NFLOG.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_RATEEST.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_SECMARK.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TEE.ko 2024-03-16T17:45:40.928 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TPROXY.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TRACE.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_addrtype.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_bpf.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cgroup.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cluster.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_comment.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connbytes.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connlabel.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connlimit.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connmark.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_conntrack.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cpu.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_dccp.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_devgroup.ko 2024-03-16T17:45:40.929 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_dscp.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ecn.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_esp.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_hashlimit.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_helper.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_hl.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ipcomp.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_iprange.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ipvs.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_l2tp.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_length.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_limit.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_mac.ko 2024-03-16T17:45:40.930 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_mark.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_multiport.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_nat.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_nfacct.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_osf.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_owner.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_physdev.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_pkttype.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_policy.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_quota.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_rateest.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_realm.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_recent.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_sctp.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_set.ko 2024-03-16T17:45:40.931 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_socket.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_state.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_statistic.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_string.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_tcpmss.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_time.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_u32.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netlink 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netlink/netlink_diag.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/nsh 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/nsh/nsh.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/openvswitch.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-geneve.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-gre.ko 2024-03-16T17:45:40.932 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-vxlan.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/packet 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/packet/af_packet_diag.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds_rdma.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds_tcp.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_csum.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_gact.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_mirred.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_nat.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_pedit.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_police.ko 2024-03-16T17:45:40.933 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_simple.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_skbedit.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_vlan.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_basic.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_bpf.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_flow.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_fw.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_route.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_u32.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_cmp.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_ipset.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_meta.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_nbyte.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_text.ko 2024-03-16T17:45:40.934 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_u32.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_choke.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_codel.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_drr.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_fq.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_gred.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_hfsc.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_hhf.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_htb.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_ingress.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_mqprio.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_mqprio_lib.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_multiq.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_netem.ko 2024-03-16T17:45:40.935 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_pie.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_plug.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_prio.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_qfq.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_red.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_sfb.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_sfq.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_tbf.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_teql.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp/sctp.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp/sctp_diag.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-16T17:45:40.936 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/sunrpc.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/xprtrdma 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc/diag.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc/tipc.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/unix 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/unix/unix_diag.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-16T17:45:40.937 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/xfrm 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/encrypted-keys 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/trusted-keys 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/trusted-keys/trusted.ko 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt/lib 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt/lib/irqbypass.ko 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.alias 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.alias.bin 2024-03-16T17:45:40.938 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.alias.bin 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.bin 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.modinfo 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.dep 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.dep.bin 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.devname 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.order 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.softdep 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.symbols 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.symbols.bin 2024-03-16T17:45:40.939 INFO:teuthology.orchestra.run.smithi090.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/vmlinuz 2024-03-16T17:45:40.943 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:40.943 DEBUG:teuthology.orchestra.run.smithi090:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout:postinstall scriptlet (using /bin/sh): 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout: /usr/bin/kernel-install add 6.8.0-rc7-g3845b7ac715a /lib/modules/6.8.0-rc7-g3845b7ac715a/vmlinuz 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout:for file in vmlinuz System.map config; do 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-g3845b7ac715a/${file}" "/boot/${file}-6.8.0-rc7-g3845b7ac715a"; then 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout: cp "/lib/modules/6.8.0-rc7-g3845b7ac715a/${file}" "/boot/${file}-6.8.0-rc7-g3845b7ac715a" 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout: fi 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout:done 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-16T17:45:40.966 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-16T17:45:40.967 INFO:teuthology.orchestra.run.smithi090.stdout:new-kernel-pkg --remove 6.8.0-rc7-g3845b7ac715a --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-g3845b7ac715a.img 2024-03-16T17:45:40.967 INFO:teuthology.orchestra.run.smithi090.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-16T17:45:40.967 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-install remove 6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:40.967 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-03-16T17:45:40.967 INFO:teuthology.orchestra.run.smithi090.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-16T17:45:40.967 INFO:teuthology.orchestra.run.smithi090.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-16T17:45:40.967 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:40.967 INFO:teuthology.orchestra.run.smithi090.stdout:fi 2024-03-16T17:45:40.968 DEBUG:teuthology.orchestra.run.smithi090:> sudo rpm -qi grub2-tools 2024-03-16T17:45:41.042 INFO:teuthology.orchestra.run.smithi090.stdout:Name : grub2-tools 2024-03-16T17:45:41.042 INFO:teuthology.orchestra.run.smithi090.stdout:Epoch : 1 2024-03-16T17:45:41.042 INFO:teuthology.orchestra.run.smithi090.stdout:Version : 2.06 2024-03-16T17:45:41.042 INFO:teuthology.orchestra.run.smithi090.stdout:Release : 68.el9 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Architecture: x86_64 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Group : Unspecified 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Size : 8264254 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:License : GPLv3+ 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Packager : builder@centos.org 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Vendor : CentOS 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Summary : Support tools for GRUB. 2024-03-16T17:45:41.043 INFO:teuthology.orchestra.run.smithi090.stdout:Description : 2024-03-16T17:45:41.044 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:45:41.044 INFO:teuthology.orchestra.run.smithi090.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-16T17:45:41.044 INFO:teuthology.orchestra.run.smithi090.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-16T17:45:41.044 INFO:teuthology.orchestra.run.smithi090.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-16T17:45:41.044 INFO:teuthology.orchestra.run.smithi090.stdout:hardware devices. 2024-03-16T17:45:41.044 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:45:41.044 INFO:teuthology.orchestra.run.smithi090.stdout:This subpackage provides tools for support of all platforms. 2024-03-16T17:45:41.045 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-16T17:45:41.045 INFO:teuthology.task.kernel:Updating grub on smithi090 to boot 6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:41.045 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-16T17:45:41.765 INFO:teuthology.orchestra.run.smithi090.stderr:Generating grub configuration file ... 2024-03-16T17:45:43.714 INFO:teuthology.orchestra.run.smithi090.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-16T17:45:43.734 INFO:teuthology.orchestra.run.smithi090.stderr:done 2024-03-16T17:45:43.737 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-03-16T17:45:43.767 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.nhUz94G6nM 2024-03-16T17:45:43.767 DEBUG:teuthology.orchestra.run.smithi090:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nhUz94G6nM 2024-03-16T17:45:43.836 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.nhUz94G6nM 2024-03-16T17:45:43.992 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.nhUz94G6nM is 6KB 2024-03-16T17:45:44.040 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.nhUz94G6nM 2024-03-16T17:45:44.054 DEBUG:teuthology.orchestra.run.smithi090:> sudo /bin/ls /boot/loader/entries || true 2024-03-16T17:45:44.122 INFO:teuthology.orchestra.run.smithi090.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-16T17:45:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-16T17:45:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-16T17:45:44.123 INFO:teuthology.orchestra.run.smithi090.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-g3845b7ac715a.conf 2024-03-16T17:45:44.124 DEBUG:teuthology.orchestra.run.smithi090:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:44.282 DEBUG:teuthology.orchestra.run.smithi090:> sudo shutdown -r now 2024-03-16T17:45:48.644 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi082.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-16T17:45:48.645 DEBUG:teuthology.orchestra.run.smithi082:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-16T17:45:48.713 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:48.713 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/System.map 2024-03-16T17:45:48.713 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/config 2024-03-16T17:45:48.713 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-16T17:45:48.714 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu 2024-03-16T17:45:48.715 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu/mce 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/arch/x86/kvm/kvm.ko 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/ansi_cprng.ko 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/anubis.ko 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/arc4.ko 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_memcpy.ko 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_pq.ko 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-16T17:45:48.716 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_tx.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/async_xor.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/async_tx/raid6test.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/authenc.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/authencesn.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blake2b_generic.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blowfish_common.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/blowfish_generic.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/camellia_generic.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast5_generic.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast6_generic.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cast_common.ko 2024-03-16T17:45:48.717 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/ccm.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cmac.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crc32_generic.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crypto_engine.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/crypto_user.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/cts.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/des_generic.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/dh_generic.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/echainiv.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/essiv.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/fcrypt.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/khazad.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/lz4.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/lz4hc.ko 2024-03-16T17:45:48.718 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/md4.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/michael_mic.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/pcbc.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/pcrypt.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/rmd160.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/seed.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/serpent_generic.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/tcrypt.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/tea.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/twofish_common.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/twofish_generic.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/vmac.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/wp512.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xcbc.ko 2024-03-16T17:45:48.719 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xor.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/crypto/xxhash_generic.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/acpi_pad.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/custom_method.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/ec_sys.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/platform_profile.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/sbs.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/sbshc.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/acpi/video.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/acard-ahci.ko 2024-03-16T17:45:48.720 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/ahci_platform.ko 2024-03-16T17:45:48.721 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/ata_generic.ko 2024-03-16T17:45:48.721 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/libahci_platform.ko 2024-03-16T17:45:48.721 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_acpi.ko 2024-03-16T17:45:48.721 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ali.ko 2024-03-16T17:45:48.721 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_amd.ko 2024-03-16T17:45:48.721 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_artop.ko 2024-03-16T17:45:48.721 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_atiixp.ko 2024-03-16T17:45:48.721 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_atp867x.ko 2024-03-16T17:45:48.721 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cmd640.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cmd64x.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_cypress.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_efar.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt366.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt37x.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_it8213.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_it821x.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_jmicron.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_marvell.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_mpiix.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_netcell.ko 2024-03-16T17:45:48.722 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ninja32.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ns87410.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_ns87415.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_oldpiix.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_opti.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_optidma.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pcmcia.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_piccolo.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_rdc.ko 2024-03-16T17:45:48.723 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sch.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_serverworks.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sil680.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sis.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_sl82c105.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_triflex.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pata_via.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/pdc_adma.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_inic162x.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_mv.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_nv.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_promise.ko 2024-03-16T17:45:48.724 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_qstor.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sil.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sil24.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sis.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_svw.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_sx4.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_uli.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_via.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ata/sata_vsc.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/auxdisplay 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/auxdisplay/ks0108.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-16T17:45:48.725 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/bcma 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/bcma/bcma.ko 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/aoe 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/aoe/aoe.ko 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/brd.ko 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/drbd 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/drbd/drbd.ko 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/floppy.ko 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/loop.ko 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/mtip32xx 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/nbd.ko 2024-03-16T17:45:48.726 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/null_blk 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/null_blk/null_blk.ko 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/pktcdvd.ko 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/rbd.ko 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/virtio_blk.ko 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkback 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/xen-blkfront.ko 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/zram 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/block/zram/zram.ko 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hangcheck-timer.ko 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-16T17:45:48.727 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/via-rng.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/lp.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/mwave 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/mwave/mwave.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/ppdev.ko 2024-03-16T17:45:48.728 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tlclk.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/uv_mmtimer.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/char/virtio_console.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-16T17:45:48.729 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/ccp/ccp.ko 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_common 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/padlock-aes.ko 2024-03-16T17:45:48.730 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/padlock-sha.ko 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/virtio 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dca 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dca/dca.ko 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/devfreq 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/dw 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/ioat 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/amd64_edac.ko 2024-03-16T17:45:48.731 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/e752x_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/edac_mce_amd.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i3000_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i3200_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i5100_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i5400_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i7300_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i7core_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/i82975x_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/ie31200_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/sb_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/edac/x38_edac.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware/edd.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-16T17:45:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio/gpio-ich.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-appleir.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-aureal.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-axff.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-dr.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-elecom.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-elo.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-emsff.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gaff.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gt683r.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-gyration.ko 2024-03-16T17:45:48.733 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-holtekff.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-hyperv.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-icade.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-keytouch.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-kye.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-lcpower.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-led.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-lenovo.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-multitouch.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-ortek.ko 2024-03-16T17:45:48.734 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-penmount.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-petalynx.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-picolcd.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-pl.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-plantronics.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-primax.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-rmi.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-common.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-16T17:45:48.735 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-roccat.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-saitek.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-samsung.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sjoy.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sony.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-speedlink.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-steelseries.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-sunplus.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-tivo.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-tmff.ko 2024-03-16T17:45:48.736 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-topseed.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-twinhan.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-uclogic.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-waltop.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-wiimote.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-xinmo.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-zpff.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/hid-zydacron.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/uhid.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hid/wacom.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_balloon.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_utils.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hv/hv_vmbus.ko 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon 2024-03-16T17:45:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/abituguru.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/abituguru3.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ad7414.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ad7418.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adc128d818.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1025.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1026.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1029.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm1031.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adm9240.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ads7828.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7410.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7411.ko 2024-03-16T17:45:48.738 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7462.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7470.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7475.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/adt7x10.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/amc6821.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/applesmc.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asb100.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asc7621.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/atxp1.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/coretemp.ko 2024-03-16T17:45:48.739 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/dme1737.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ds1621.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ds620.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/emc1403.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/emc6w201.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f71805f.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f71882fg.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/f75375s.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/fam15h_power.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/fschmd.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/g760a.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/g762.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/gl518sm.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/gl520sm.ko 2024-03-16T17:45:48.740 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/i5500_temp.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/i5k_amb.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ibmaem.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ibmpex.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ina209.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ina2xx.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/it87.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/k10temp.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/k8temp.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lineage-pem.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm63.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm73.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm75.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm77.ko 2024-03-16T17:45:48.741 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm78.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm80.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm83.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm85.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm87.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm90.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm92.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm93.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95234.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95241.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/lm95245.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc2945.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4151.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4215.ko 2024-03-16T17:45:48.742 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4222.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4245.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4260.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/ltc4261.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max16065.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max1619.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max1668.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max197.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6639.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6650.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/max6697.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/mcp3021.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6683.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6775-core.ko 2024-03-16T17:45:48.743 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct6775.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/nct7802.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pc87360.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pc87427.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pcf8591.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-16T17:45:48.744 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/powr1220.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch5627.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch5636.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sht15.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sht21.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/shtc1.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/sis5595.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47b397.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47m1.ko 2024-03-16T17:45:48.745 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/smsc47m192.ko 2024-03-16T17:45:48.746 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/thmc50.ko 2024-03-16T17:45:48.746 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp102.ko 2024-03-16T17:45:48.746 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp103.ko 2024-03-16T17:45:48.746 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp401.ko 2024-03-16T17:45:48.746 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/tmp421.ko 2024-03-16T17:45:48.746 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/via-cputemp.ko 2024-03-16T17:45:48.746 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/via686a.ko 2024-03-16T17:45:48.746 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/vt1211.ko 2024-03-16T17:45:48.746 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/vt8231.ko 2024-03-16T17:45:48.746 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83627ehf.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83627hf.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83781d.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83791d.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83792d.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83793.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83795.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83l785ts.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/hwmon/w83l786ng.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-16T17:45:48.747 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-16T17:45:48.748 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-dev.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-mux.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-smbus.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/i2c/i2c-stub.ko 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband 2024-03-16T17:45:48.749 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_core.ko 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/cxgb4 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx4 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx5 2024-03-16T17:45:48.750 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mthca 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/ipoib 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/iser 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/isert 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srp 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srpt 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input 2024-03-16T17:45:48.751 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/gameport/gameport.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/joydev.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/keyboard 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/appletouch.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/bcm5974.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/cyapatp.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/sermouse.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-16T17:45:48.752 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/rmi4 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/altera_ps2.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/arc_ps2.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/serio/serio_raw.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/input/sparse-keymap.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-blinkm.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-lm3530.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-lp3944.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/leds-ss4200.ko 2024-03-16T17:45:48.753 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/bcache 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/bcache/bcache.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-bio-prison.ko 2024-03-16T17:45:48.754 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-cache-smq.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-cache.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-crypt.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-delay.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-flakey.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-log-userspace.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-multipath.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-queue-length.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-raid.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-round-robin.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-service-time.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-switch.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-thin-pool.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/dm-verity.ko 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/persistent-data 2024-03-16T17:45:48.755 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid0.ko 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid1.ko 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid10.ko 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/md/raid456.ko 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptbase.ko 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptctl.ko 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptfc.ko 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptlan.ko 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptsas.ko 2024-03-16T17:45:48.756 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptscsih.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/message/fusion/mptspi.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/lpc_ich.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/lpc_sch.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/mfd-core.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/sm501.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/viperboard.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/vx855.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/mfd/wl1273-core.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/altera-stapl 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/apds9802als.ko 2024-03-16T17:45:48.757 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/apds990x.ko 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/bh1770glc.ko 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/cb710 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/cb710/cb710.ko 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/at24.ko 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/eeprom/max6875.ko 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/enclosure.ko 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/hpilo.ko 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/isl29003.ko 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/isl29020.ko 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d 2024-03-16T17:45:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei-me.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei-txe.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/mei/mei.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-gru 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tifm_7xx1.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tifm_core.ko 2024-03-16T17:45:48.759 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/tsl2550.ko 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_balloon.ko 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_vmci 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/bonding 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/bonding/bonding.ko 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/b53 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6xxx 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-16T17:45:48.760 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/dummy.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/eql.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/adaptec 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-16T17:45:48.761 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/agere 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/alteon 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/altera 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/alx 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-16T17:45:48.762 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/genet 2024-03-16T17:45:48.763 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade/bna 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-16T17:45:48.764 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco/enic 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink 2024-03-16T17:45:48.765 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/dnet.ko 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex/benet 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ethoc.ko 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/fealnx.ko 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000e 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-16T17:45:48.766 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/fm10k 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/i40e 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/iavf 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igb 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igbvf 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/jme.ko 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell 2024-03-16T17:45:48.767 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/micrel 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-16T17:45:48.768 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-16T17:45:48.769 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi 2024-03-16T17:45:48.769 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-16T17:45:48.769 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-16T17:45:48.769 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/neterion 2024-03-16T17:45:48.769 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-16T17:45:48.769 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/nvidia 2024-03-16T17:45:48.769 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-16T17:45:48.770 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines 2024-03-16T17:45:48.770 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-16T17:45:48.770 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-16T17:45:48.770 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic 2024-03-16T17:45:48.770 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-16T17:45:48.770 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rdc 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rocker 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sfc 2024-03-16T17:45:48.771 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/silan 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-16T17:45:48.772 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/tehuti 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ti 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet 2024-03-16T17:45:48.773 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/xircom 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/geneve.ko 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/hyperv 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ieee802154 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ifb.ko 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ipvlan 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/macvlan.ko 2024-03-16T17:45:48.774 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/macvtap.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/mii.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/net_failover.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/netconsole.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/nlmon.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy 2024-03-16T17:45:48.775 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/amd.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/bcm87xx.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/broadcom.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/cicada.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/davicom.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/dp83640.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/et1011c.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/icplus.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/lxt.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/marvell.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/micrel.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/national.ko 2024-03-16T17:45:48.776 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/phylink.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/qsemi.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/realtek.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/smsc.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/ste10Xp.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/phy/vitesse.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_async.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pppoe.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pppox.ko 2024-03-16T17:45:48.777 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/ppp/pptp.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip/slhc.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/slip/slip.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/sungem_phy.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/tap.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_random.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/tun.ko 2024-03-16T17:45:48.778 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/veth.ko 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/virtio_net.ko 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vmxnet3 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vxlan 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/vxlan/vxlan.ko 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netback 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/net/xen-netfront.ko 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ntb 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ntb/ntb.ko 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host/nvme-core.ko 2024-03-16T17:45:48.779 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/nvme/host/nvme.ko 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport.ko 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_cs.ko 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_pc.ko 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/parport/parport_serial.ko 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/hotplug 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/pcie 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pci/xen-pcifront.ko 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/i82092.ko 2024-03-16T17:45:48.780 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/pd6729.ko 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl/intel 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/acerhdf.ko 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-16T17:45:48.781 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/hdaps.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/intel_ips.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-16T17:45:48.782 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/platform/x86/wmi.ko 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/clients/pps_parport.ko 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/pps/pps_core.ko 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp 2024-03-16T17:45:48.783 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp/ptp.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ptp/ptp_kvm.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-em3027.ko 2024-03-16T17:45:48.784 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-max6900.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-16T17:45:48.785 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/rtc/rtc-x1205.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-9xxx.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-sas.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/3w-xxxx.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/BusLogic.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/a100u2w.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aacraid 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-16T17:45:48.786 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/advansys.ko 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/am53c974.ko 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/arcmsr 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/atp870u.ko 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/be2iscsi 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bfa 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bfa/bfa.ko 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2fc 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-16T17:45:48.787 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2i 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ch.ko 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/csiostor 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/dc395x.ko 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-16T17:45:48.788 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/dmx3191d.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esas2r 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/esp_scsi.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fnic 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/fnic/fnic.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hpsa.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hptiop.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/hv_storvsc.ko 2024-03-16T17:45:48.789 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/initio.ko 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ipr.ko 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ips.ko 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/isci 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/isci/isci.ko 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libfc 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libfc/libfc.ko 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libiscsi.ko 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libsas 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/libsas/libsas.ko 2024-03-16T17:45:48.790 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/lpfc 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mpt3sas 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvsas 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/mvumi.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pm8001 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/pmcraid.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla1280.ko 2024-03-16T17:45:48.791 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla4xxx 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/raid_class.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_debug.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/ses.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/st.ko 2024-03-16T17:45:48.792 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/stex.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/sym53c8xx_2 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/virtio_scsi.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/scsi/wd719x.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ssb 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ssb/ssb.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/iscsi 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/loopback 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_file.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_iblock.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_mod.ko 2024-03-16T17:45:48.793 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_pscsi.ko 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/target_core_user.ko 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/tcm_fc 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-16T17:45:48.794 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/ipwireless 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-16T17:45:48.795 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/n_gsm.ko 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/n_hdlc.ko 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/nozomi.ko 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/8250 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/arc_uart.ko 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/jsm 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/core 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/host 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-16T17:45:48.796 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio.ko 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_aec.ko 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_cif.ko 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_pci_generic.ko 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/uio/uio_sercos3.ko 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/cdc-acm.ko 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/usblp.ko 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/class/usbtmc.ko 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/host 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-16T17:45:48.797 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image/mdc800.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/image/microtek.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/adutux.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/appledisplay.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/emi26.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/emi62.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/ezusb.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/idmouse.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/iowarrior.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/ldusb.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/legousbtower.ko 2024-03-16T17:45:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/sisusbvga 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usb3503.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usblcd.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/uss720.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/misc/yurex.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/phy 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/phy/phy-generic.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/aircable.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ark3116.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ch341.ko 2024-03-16T17:45:48.799 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cp210x.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cyberjack.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/empeg.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/io_ti.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ipaq.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ipw.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ir-usb.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/keyspan.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-16T17:45:48.800 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mct_u232.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mos7720.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/mos7840.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/navman.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/omninet.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/opticon.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/option.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/oti6858.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/pl2303.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/qcaux.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/qcserial.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/quatech2.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/safe_serial.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/sierra.ko 2024-03-16T17:45:48.801 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ssu100.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/symbolserial.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb_debug.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/visor.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/whiteheat.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/uas.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-16T17:45:48.802 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-karma.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/storage/usb-storage.ko 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip 2024-03-16T17:45:48.803 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/vfio.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_net.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/vhost/vhost_scsi.ko 2024-03-16T17:45:48.804 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/apple_bl.ko 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/lcd.ko 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio.ko 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_balloon.ko 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_mmio.ko 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci.ko 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/virtio/virtio_ring.ko 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters 2024-03-16T17:45:48.805 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters/ds2482.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/masters/ds2490.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-16T17:45:48.806 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/w1/wire.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/hpwdt.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/i6300esb.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ib700wdt.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ibmasr.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/it87_wdt.ko 2024-03-16T17:45:48.807 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/machzwd.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/nv_tco.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/softdog.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/via_wdt.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/wdt_pci.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/watchdog/xen_wdt.ko 2024-03-16T17:45:48.808 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-evtchn.ko 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-gntalloc.ko 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-gntdev.ko 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-pciback 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xen-privcmd.ko 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xenfs 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/9p 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/9p/9p.ko 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/affs 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/affs/affs.ko 2024-03-16T17:45:48.809 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/befs 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/befs/befs.ko 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/binfmt_misc.ko 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/btrfs 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/btrfs/btrfs.ko 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cachefiles 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cachefiles/cachefiles.ko 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ceph 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ceph/ceph.ko 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/coda 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/coda/coda.ko 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cramfs 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/cramfs/cramfs.ko 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/dlm 2024-03-16T17:45:48.810 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/dlm/dlm.ko 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ecryptfs 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/f2fs 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/f2fs/f2fs.ko 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/fat.ko 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/msdos.ko 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fat/vfat.ko 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse/cuse.ko 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/fuse/fuse.ko 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/gfs2 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/gfs2/gfs2.ko 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfs 2024-03-16T17:45:48.811 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfs/hfs.ko 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfsplus 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/hfsplus/hfsplus.ko 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/isofs 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/isofs/isofs.ko 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/jfs 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/jfs/jfs.ko 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/lockd 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/lockd/lockd.ko 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/minix 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/minix/minix.ko 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/netfs 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/netfs/netfs.ko 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/blocklayout 2024-03-16T17:45:48.812 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/filelayout 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/flexfilelayout 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfs.ko 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfsv3.ko 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs/nfsv4.ko 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common/grace.ko 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfs_common/nfs_acl.ko 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfsd 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nfsd/nfsd.ko 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nilfs2 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nilfs2/nilfs2.ko 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls 2024-03-16T17:45:48.813 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-celtic.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-centeuro.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-croatian.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-cyrillic.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-gaelic.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-greek.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-iceland.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-inuit.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-roman.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-romanian.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/mac-turkish.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1250.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1251.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp1255.ko 2024-03-16T17:45:48.814 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp737.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp775.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp850.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp852.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp855.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp857.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp860.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp861.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp862.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp863.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp864.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp865.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp866.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp869.ko 2024-03-16T17:45:48.815 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp874.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp932.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp936.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp949.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_cp950.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_euc-jp.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-1.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-13.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-14.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-15.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-2.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-3.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-4.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-5.ko 2024-03-16T17:45:48.816 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-6.ko 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-7.ko 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_iso8859-9.ko 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-r.ko 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-ru.ko 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_koi8-u.ko 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/nls/nls_utf8.ko 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/cluster 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlm 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlmfs 2024-03-16T17:45:48.817 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2.ko 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/overlayfs 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/overlayfs/overlay.ko 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/pstore 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/pstore/ramoops.ko 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/reiserfs 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/reiserfs/reiserfs.ko 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/romfs 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/romfs/romfs.ko 2024-03-16T17:45:48.818 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/client 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/client/cifs.ko 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common/cifs_arc4.ko 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/smb/common/cifs_md4.ko 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/squashfs 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/squashfs/squashfs.ko 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/sysv 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/sysv/sysv.ko 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/udf 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/udf/udf.ko 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ufs 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/ufs/ufs.ko 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/xfs 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/fs/xfs/xfs.ko 2024-03-16T17:45:48.819 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/rcu 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/rcu/rcutorture.ko 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/torture.ko 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/trace 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/asn1_encoder.ko 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crc-itu-t.ko 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crc8.ko 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto/libarc4.ko 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/crypto/libdes.ko 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/libcrc32c.ko 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lru_cache.ko 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4 2024-03-16T17:45:48.820 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4/lz4_compress.ko 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/lz4/lz4hc_compress.ko 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/math 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/math/cordic.ko 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/raid6 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/raid6/raid6_pq.ko 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/reed_solomon 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_bm.ko 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_fsm.ko 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/ts_kmp.ko 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/zstd 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/lib/zstd/zstd_compress.ko 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/mm 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/mm/hwpoison-inject.ko 2024-03-16T17:45:48.821 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/6lowpan.ko 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_dest.ko 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_fragment.ko 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_hop.ko 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_mobility.ko 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_routing.ko 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/6lowpan/nhc_udp.ko 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/garp.ko 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/mrp.ko 2024-03-16T17:45:48.822 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/p8022.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/psnap.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/802/stp.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/8021q 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/8021q/8021q.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_fd.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_rdma.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/9p/9pnet_virtio.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/batman-adv 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/batman-adv/batman-adv.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/br_netfilter.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/bridge.ko 2024-03-16T17:45:48.823 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-16T17:45:48.824 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/ebtables.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ceph 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ceph/libceph.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core/failover.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/core/pktgen.ko 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp 2024-03-16T17:45:48.825 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp.ko 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_diag.ko 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_ipv4.ko 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dccp/dccp_ipv6.ko 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dns_resolver 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dns_resolver/dns_resolver.ko 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/dsa_core.ko 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_brcm.ko 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_dsa.ko 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_none.ko 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/dsa/tag_trailer.ko 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154 2024-03-16T17:45:48.826 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/6lowpan 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/ieee802154.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ah4.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/esp4.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/fou.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/gre.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/inet_diag.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_gre.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_tunnel.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ip_vti.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ipcomp.ko 2024-03-16T17:45:48.827 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/ipip.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-16T17:45:48.828 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_bic.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_dctcp.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_diag.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_highspeed.ko 2024-03-16T17:45:48.829 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_htcp.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_hybla.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_illinois.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_lp.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_scalable.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_vegas.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_veno.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_westwood.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tcp_yeah.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/tunnel4.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/udp_diag.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/udp_tunnel.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-16T17:45:48.830 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ah6.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/esp6.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/fou6.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_tunnel.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ip6_vti.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/ipcomp6.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-16T17:45:48.831 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-16T17:45:48.832 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/sit.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/tunnel6.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/key 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/key/af_key.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_core.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_eth.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ip.ko 2024-03-16T17:45:48.833 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ip6.ko 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_netlink.ko 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/l2tp/l2tp_ppp.ko 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/llc 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/llc/llc.ko 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mac802154 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mac802154/mac802154.ko 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mpls 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/mpls/mpls_gso.ko 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set.ko 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-16T17:45:48.834 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-16T17:45:48.835 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conncount.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack.ko 2024-03-16T17:45:48.836 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-16T17:45:48.837 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_flow_table.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_log_syslog.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_irc.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_sip.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nf_tables.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_log.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-16T17:45:48.838 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_chain_nat.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_compat.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_ct.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_fib.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_hash.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_limit.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_log.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_masq.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_nat.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_numgen.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_queue.ko 2024-03-16T17:45:48.839 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_quota.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_redir.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_reject.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_reject_inet.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_socket.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/nft_tunnel.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_AUDIT.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_CT.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_DSCP.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_HL.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_HMARK.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_LED.ko 2024-03-16T17:45:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_LOG.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NETMAP.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NFLOG.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_RATEEST.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_SECMARK.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TEE.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TPROXY.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_TRACE.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_addrtype.ko 2024-03-16T17:45:48.841 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_bpf.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cgroup.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cluster.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_comment.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connbytes.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connlabel.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connlimit.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_connmark.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_conntrack.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_cpu.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_dccp.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_devgroup.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_dscp.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ecn.ko 2024-03-16T17:45:48.842 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_esp.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_hashlimit.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_helper.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_hl.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ipcomp.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_iprange.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_ipvs.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_l2tp.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_length.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_limit.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_mac.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_mark.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_multiport.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_nat.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_nfacct.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_osf.ko 2024-03-16T17:45:48.843 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_owner.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_physdev.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_pkttype.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_policy.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_quota.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_rateest.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_realm.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_recent.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_sctp.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_set.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_socket.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_state.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_statistic.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_string.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_tcpmss.ko 2024-03-16T17:45:48.844 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_time.ko 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netfilter/xt_u32.ko 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netlink 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/netlink/netlink_diag.ko 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/nsh 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/nsh/nsh.ko 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/openvswitch.ko 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-geneve.ko 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-gre.ko 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/openvswitch/vport-vxlan.ko 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/packet 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/packet/af_packet_diag.ko 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds 2024-03-16T17:45:48.845 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds_rdma.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/rds/rds_tcp.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_csum.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_gact.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_mirred.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_nat.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_pedit.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_police.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_simple.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_skbedit.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/act_vlan.ko 2024-03-16T17:45:48.846 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_basic.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_bpf.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_flow.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_fw.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_route.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/cls_u32.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_cmp.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_ipset.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_meta.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_nbyte.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_text.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/em_u32.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_choke.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_codel.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_drr.ko 2024-03-16T17:45:48.847 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_fq.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_gred.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_hfsc.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_hhf.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_htb.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_ingress.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_mqprio.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_mqprio_lib.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_multiq.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_netem.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_pie.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_plug.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_prio.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_qfq.ko 2024-03-16T17:45:48.848 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_red.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_sfb.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_sfq.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_tbf.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sched/sch_teql.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp/sctp.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sctp/sctp_diag.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/sunrpc.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/xprtrdma 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc 2024-03-16T17:45:48.849 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc/diag.ko 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/tipc/tipc.ko 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/unix 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/unix/unix_diag.ko 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock.ko 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/xfrm 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/encrypted-keys 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-16T17:45:48.850 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/trusted-keys 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/security/keys/trusted-keys/trusted.ko 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt/lib 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/kernel/virt/lib/irqbypass.ko 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.alias 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.alias.bin 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.alias.bin 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.bin 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.builtin.modinfo 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.dep 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.dep.bin 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.devname 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.order 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.softdep 2024-03-16T17:45:48.851 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.symbols 2024-03-16T17:45:48.852 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/modules.symbols.bin 2024-03-16T17:45:48.852 INFO:teuthology.orchestra.run.smithi082.stdout:/lib/modules/6.8.0-rc7-g3845b7ac715a/vmlinuz 2024-03-16T17:45:48.855 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:48.855 DEBUG:teuthology.orchestra.run.smithi082:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-16T17:45:48.878 INFO:teuthology.orchestra.run.smithi082.stdout:postinstall scriptlet (using /bin/sh): 2024-03-16T17:45:48.878 INFO:teuthology.orchestra.run.smithi082.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-16T17:45:48.879 INFO:teuthology.orchestra.run.smithi082.stdout: /usr/bin/kernel-install add 6.8.0-rc7-g3845b7ac715a /lib/modules/6.8.0-rc7-g3845b7ac715a/vmlinuz 2024-03-16T17:45:48.879 INFO:teuthology.orchestra.run.smithi082.stdout:fi 2024-03-16T17:45:48.879 INFO:teuthology.orchestra.run.smithi082.stdout:for file in vmlinuz System.map config; do 2024-03-16T17:45:48.879 INFO:teuthology.orchestra.run.smithi082.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-g3845b7ac715a/${file}" "/boot/${file}-6.8.0-rc7-g3845b7ac715a"; then 2024-03-16T17:45:48.879 INFO:teuthology.orchestra.run.smithi082.stdout: cp "/lib/modules/6.8.0-rc7-g3845b7ac715a/${file}" "/boot/${file}-6.8.0-rc7-g3845b7ac715a" 2024-03-16T17:45:48.879 INFO:teuthology.orchestra.run.smithi082.stdout: fi 2024-03-16T17:45:48.879 INFO:teuthology.orchestra.run.smithi082.stdout:done 2024-03-16T17:45:48.879 INFO:teuthology.orchestra.run.smithi082.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-16T17:45:48.880 INFO:teuthology.orchestra.run.smithi082.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-16T17:45:48.880 INFO:teuthology.orchestra.run.smithi082.stdout:new-kernel-pkg --remove 6.8.0-rc7-g3845b7ac715a --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-g3845b7ac715a.img 2024-03-16T17:45:48.880 INFO:teuthology.orchestra.run.smithi082.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-16T17:45:48.880 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-install remove 6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:48.880 INFO:teuthology.orchestra.run.smithi082.stdout:fi 2024-03-16T17:45:48.880 INFO:teuthology.orchestra.run.smithi082.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-16T17:45:48.880 INFO:teuthology.orchestra.run.smithi082.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-16T17:45:48.881 INFO:teuthology.orchestra.run.smithi082.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:48.881 INFO:teuthology.orchestra.run.smithi082.stdout:fi 2024-03-16T17:45:48.881 DEBUG:teuthology.orchestra.run.smithi082:> sudo rpm -qi grub2-tools 2024-03-16T17:45:48.952 INFO:teuthology.orchestra.run.smithi082.stdout:Name : grub2-tools 2024-03-16T17:45:48.952 INFO:teuthology.orchestra.run.smithi082.stdout:Epoch : 1 2024-03-16T17:45:48.952 INFO:teuthology.orchestra.run.smithi082.stdout:Version : 2.06 2024-03-16T17:45:48.952 INFO:teuthology.orchestra.run.smithi082.stdout:Release : 68.el9 2024-03-16T17:45:48.952 INFO:teuthology.orchestra.run.smithi082.stdout:Architecture: x86_64 2024-03-16T17:45:48.952 INFO:teuthology.orchestra.run.smithi082.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-16T17:45:48.952 INFO:teuthology.orchestra.run.smithi082.stdout:Group : Unspecified 2024-03-16T17:45:48.952 INFO:teuthology.orchestra.run.smithi082.stdout:Size : 8264254 2024-03-16T17:45:48.952 INFO:teuthology.orchestra.run.smithi082.stdout:License : GPLv3+ 2024-03-16T17:45:48.952 INFO:teuthology.orchestra.run.smithi082.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:Packager : builder@centos.org 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:Vendor : CentOS 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:Summary : Support tools for GRUB. 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:Description : 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:hardware devices. 2024-03-16T17:45:48.953 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:45:48.954 INFO:teuthology.orchestra.run.smithi082.stdout:This subpackage provides tools for support of all platforms. 2024-03-16T17:45:48.954 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-16T17:45:48.954 INFO:teuthology.task.kernel:Updating grub on smithi082 to boot 6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:48.955 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-16T17:45:49.567 INFO:teuthology.orchestra.run.smithi082.stderr:Generating grub configuration file ... 2024-03-16T17:45:50.994 INFO:teuthology.orchestra.run.smithi082.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-16T17:45:51.014 INFO:teuthology.orchestra.run.smithi082.stderr:done 2024-03-16T17:45:51.016 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-03-16T17:45:51.031 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.7q9vCcfnwE 2024-03-16T17:45:51.032 DEBUG:teuthology.orchestra.run.smithi082:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7q9vCcfnwE 2024-03-16T17:45:51.099 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.7q9vCcfnwE 2024-03-16T17:45:51.230 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.7q9vCcfnwE is 6KB 2024-03-16T17:45:51.280 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.7q9vCcfnwE 2024-03-16T17:45:51.294 DEBUG:teuthology.orchestra.run.smithi082:> sudo /bin/ls /boot/loader/entries || true 2024-03-16T17:45:51.382 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-16T17:45:51.383 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-16T17:45:51.383 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-16T17:45:51.383 INFO:teuthology.orchestra.run.smithi082.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-g3845b7ac715a.conf 2024-03-16T17:45:51.384 DEBUG:teuthology.orchestra.run.smithi082:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-g3845b7ac715a 2024-03-16T17:45:51.510 DEBUG:teuthology.orchestra.run.smithi082:> sudo shutdown -r now 2024-03-16T17:46:14.307 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-16T17:46:14.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-16T17:46:14.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:46:21.520 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-16T17:46:21.520 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-16T17:46:21.521 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:46:32.863 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-16T17:46:40.030 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-03-16T17:46:41.866 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-16T17:46:41.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:46:44.926 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-16T17:46:49.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-16T17:46:49.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:46:52.098 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.82 2024-03-16T17:46:56.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-16T17:46:56.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:47:00.002 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-16T17:47:04.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-16T17:47:04.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:47:15.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-16T17:47:15.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:47:33.539 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.90 2024-03-16T17:47:35.954 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-03-16T17:47:36.624 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-16T17:47:36.624 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc7-g3845b7ac715a"... 2024-03-16T17:47:36.624 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-03-16T17:47:36.662 INFO:teuthology.orchestra.run.smithi082.stdout:6.8.0-rc7-g3845b7ac715a 2024-03-16T17:47:36.662 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-g3845b7ac715a vs 6.8.0-rc7-g3845b7ac715a 2024-03-16T17:47:36.662 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T17:47:36.662 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-16T17:47:37.664 DEBUG:teuthology.parallel:result is None 2024-03-16T17:47:51.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-16T17:47:51.541 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:47:52.170 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-03-16T17:47:53.024 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-16T17:47:53.024 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-rc7-g3845b7ac715a"... 2024-03-16T17:47:53.024 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-03-16T17:47:53.069 INFO:teuthology.orchestra.run.smithi090.stdout:6.8.0-rc7-g3845b7ac715a 2024-03-16T17:47:53.070 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-g3845b7ac715a vs 6.8.0-rc7-g3845b7ac715a 2024-03-16T17:47:53.070 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T17:47:53.070 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-16T17:47:54.071 DEBUG:teuthology.parallel:result is None 2024-03-16T17:47:54.071 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-16T17:47:54.083 INFO:teuthology.task.internal:Creating test directory... 2024-03-16T17:47:54.083 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T17:47:54.092 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T17:47:54.143 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-16T17:47:54.149 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-16T17:47:54.160 INFO:teuthology.task.internal:Creating archive directory... 2024-03-16T17:47:54.160 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T17:47:54.167 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T17:47:54.235 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-16T17:47:54.242 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-16T17:47:54.242 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T17:47:54.251 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T17:47:54.404 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T17:47:54.449 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T17:47:54.467 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T17:47:54.545 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T17:47:54.552 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-16T17:47:54.559 INFO:teuthology.task.internal:Configuring sudo... 2024-03-16T17:47:54.559 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T17:47:54.566 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T17:47:54.657 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-16T17:47:54.667 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-16T17:47:54.667 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T17:47:54.674 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T17:47:54.718 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T17:47:54.793 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T17:47:54.896 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T17:47:54.978 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T17:47:55.073 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:47:55.073 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T17:47:55.166 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T17:47:55.253 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T17:47:55.335 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T17:47:55.405 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T17:47:55.492 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:47:55.492 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T17:47:55.573 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-03-16T17:47:55.581 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-03-16T17:47:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T17:47:55.730 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T17:47:56.218 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-16T17:47:56.226 INFO:teuthology.task.internal:Starting timer... 2024-03-16T17:47:56.226 INFO:teuthology.run_tasks:Running task pcp... 2024-03-16T17:47:56.235 INFO:teuthology.run_tasks:Running task selinux... 2024-03-16T17:47:56.244 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2024-03-16T17:47:56.363 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2024-03-16T17:47:56.368 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-03-16T17:47:56.497 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-03-16T17:47:56.501 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-16T17:47:56.501 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2024-03-16T17:47:56.560 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2024-03-16T17:47:56.560 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-03-16T17:47:56.614 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-03-16T17:47:56.614 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi082.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi090.front.sepia.ceph.com': 'permissive'} 2024-03-16T17:47:56.614 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-16T17:47:56.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T17:47:56.688 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-16T17:47:56.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T17:47:56.767 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-16T17:47:56.767 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2024-03-16T17:47:56.864 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-03-16T17:47:56.984 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-16T17:47:56.994 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-16T17:47:57.001 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-16T17:47:57.120 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-16T17:47:57.137 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-16T17:47:57.138 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi082.front.sepia.ceph.com,smithi090.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-16T17:53:54.052 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi090.front.sepia.ceph.com')] 2024-03-16T17:53:54.053 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-16T17:53:54.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:53:54.204 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-03-16T17:53:54.355 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-03-16T17:53:54.356 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-16T17:53:54.356 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T17:53:54.496 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-03-16T17:53:54.650 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-03-16T17:53:54.651 INFO:teuthology.run_tasks:Running task clock... 2024-03-16T17:53:54.664 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-16T17:53:54.664 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T17:53:54.664 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T17:53:54.672 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T17:53:54.672 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T17:53:54.767 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T17:53:54.769 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-16T17:53:54.818 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T17:53:54.820 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-16T17:53:54.913 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2024-03-16T17:53:54.914 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-03-16T17:53:54.955 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2024-03-16T17:53:54.956 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-03-16T17:53:55.006 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T17:53:55.007 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-16T17:53:55.060 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T17:53:55.060 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-16T17:53:55.188 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-03-16T17:53:55.194 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-03-16T17:53:55.197 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T17:53:55.197 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-03-16T17:53:55.197 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T17:53:55.197 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T17:53:55.197 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T17:53:55.197 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T17:53:55.205 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T17:53:55.205 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-03-16T17:53:55.205 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T17:53:55.205 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T17:53:55.205 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T17:53:55.205 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T17:53:55.210 INFO:teuthology.run_tasks:Running task install... 2024-03-16T17:53:55.219 DEBUG:teuthology.task.install:project ceph 2024-03-16T17:53:55.219 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}} 2024-03-16T17:53:55.219 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'} 2024-03-16T17:53:55.219 INFO:teuthology.task.install:Using flavor: default 2024-03-16T17:53:55.225 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-16T17:53:55.225 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-16T17:53:55.225 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'tag': None, 'wait_for_package': False} 2024-03-16T17:53:55.226 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-16T17:53:55.228 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'tag': None, 'wait_for_package': False} 2024-03-16T17:53:55.228 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-16T17:53:55.372 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/ 2024-03-16T17:53:55.372 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1681.gaba3c1d2 2024-03-16T17:53:55.438 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/ 2024-03-16T17:53:55.438 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1681.gaba3c1d2 2024-03-16T17:53:55.487 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T17:53:55.487 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:53:55.488 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T17:53:55.582 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T17:53:55.582 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T17:53:55.610 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-16T17:53:55.610 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:53:55.610 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-16T17:53:55.709 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-16T17:53:55.709 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-16T17:53:55.719 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T17:53:55.844 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-16T17:53:55.849 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T17:53:55.939 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-03-16T17:53:55.944 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-03-16T17:53:55.975 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-16T17:53:56.063 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2024-03-16T17:53:56.067 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2024-03-16T17:53:56.291 INFO:teuthology.orchestra.run.smithi090.stdout:74 files removed 2024-03-16T17:53:56.329 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T17:53:56.408 INFO:teuthology.orchestra.run.smithi082.stdout:74 files removed 2024-03-16T17:53:56.446 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-16T17:53:57.312 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 173 kB/s | 83 kB 00:00 2024-03-16T17:53:57.520 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 146 kB/s | 83 kB 00:00 2024-03-16T17:53:57.805 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-16T17:53:58.074 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 22 kB/s | 10 kB 00:00 2024-03-16T17:53:58.372 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-03-16T17:53:58.580 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-03-16T17:53:59.036 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS-9 - AppStream 11 MB/s | 6.7 MB 00:00 2024-03-16T17:53:59.166 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-16T17:54:00.712 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-03-16T17:54:00.766 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.0 MB 00:00 2024-03-16T17:54:01.984 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.0 MB 00:00 2024-03-16T17:54:05.330 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 19 MB 00:02 2024-03-16T17:54:10.616 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 863 kB/s | 8.0 MB 00:09 2024-03-16T17:54:11.738 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.1 MB 00:02 2024-03-16T17:54:13.126 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-03-16T17:54:13.330 INFO:teuthology.orchestra.run.smithi082.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 15 kB 00:00 2024-03-16T17:54:18.698 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 4.0 MB/s | 21 MB 00:05 2024-03-16T17:54:24.447 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-16T17:54:25.561 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 724 kB/s | 6.1 MB 00:08 2024-03-16T17:54:27.232 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T17:54:27.234 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T17:54:27.257 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T17:54:27.259 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T17:54:27.260 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T17:54:27.262 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T17:54:27.265 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T17:54:27.268 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T17:54:27.270 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T17:54:27.347 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 15 kB 00:00 2024-03-16T17:54:27.357 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-03-16T17:54:27.374 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================ 2024-03-16T17:54:27.374 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-03-16T17:54:27.374 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================ 2024-03-16T17:54:27.374 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-03-16T17:54:27.374 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T17:54:27.374 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 6.6 k 2024-03-16T17:54:27.374 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 5.3 M 2024-03-16T17:54:27.374 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 887 k 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 145 k 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 1.5 M 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 138 k 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 1.8 M 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 7.4 M 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 49 k 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 9.8 M 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 47 M 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 264 k 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 748 k 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 218 k 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 27 k 2024-03-16T17:54:27.375 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 18 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 31 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 744 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 127 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 161 k 2024-03-16T17:54:27.376 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 321 k 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 298 k 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 99 k 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 85 k 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.1 M 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 171 k 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T17:54:27.377 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.4 M 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.1 M 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T17:54:27.378 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 20 M 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 31 k 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 2.3 M 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 248 k 2024-03-16T17:54:27.379 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 4.7 M 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 17 M 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 15 k 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 25 k 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 163 k 2024-03-16T17:54:27.380 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 498 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 5.1 M 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T17:54:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T17:54:27.382 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 45 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 131 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T17:54:27.383 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T17:54:27.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T17:54:27.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T17:54:27.386 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================ 2024-03-16T17:54:27.387 INFO:teuthology.orchestra.run.smithi082.stdout:Install 150 Packages 2024-03-16T17:54:27.388 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 24 Packages 2024-03-16T17:54:27.388 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:54:27.388 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 224 M 2024-03-16T17:54:27.388 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-03-16T17:54:28.265 INFO:teuthology.orchestra.run.smithi082.stdout:(1/174): ceph-19.0.0-1681.gaba3c1d2.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-03-16T17:54:28.665 INFO:teuthology.orchestra.run.smithi082.stdout:(2/174): ceph-base-19.0.0-1681.gaba3c1d2.el9.x8 7.5 MB/s | 5.3 MB 00:00 2024-03-16T17:54:28.724 INFO:teuthology.orchestra.run.smithi082.stdout:(3/174): ceph-immutable-object-cache-19.0.0-168 2.5 MB/s | 145 kB 00:00 2024-03-16T17:54:28.766 INFO:teuthology.orchestra.run.smithi082.stdout:(4/174): ceph-fuse-19.0.0-1681.gaba3c1d2.el9.x8 1.7 MB/s | 887 kB 00:00 2024-03-16T17:54:28.957 INFO:teuthology.orchestra.run.smithi082.stdout:(5/174): ceph-mds-19.0.0-1681.gaba3c1d2.el9.x86 9.9 MB/s | 2.3 MB 00:00 2024-03-16T17:54:29.266 INFO:teuthology.orchestra.run.smithi082.stdout:(6/174): ceph-mgr-19.0.0-1681.gaba3c1d2.el9.x86 3.0 MB/s | 1.5 MB 00:00 2024-03-16T17:54:29.375 INFO:teuthology.orchestra.run.smithi082.stdout:(7/174): ceph-mon-19.0.0-1681.gaba3c1d2.el9.x86 11 MB/s | 4.7 MB 00:00 2024-03-16T17:54:30.216 INFO:teuthology.orchestra.run.smithi082.stdout:(8/174): ceph-radosgw-19.0.0-1681.gaba3c1d2.el9 12 MB/s | 9.8 MB 00:00 2024-03-16T17:54:30.266 INFO:teuthology.orchestra.run.smithi082.stdout:(9/174): ceph-selinux-19.0.0-1681.gaba3c1d2.el9 515 kB/s | 25 kB 00:00 2024-03-16T17:54:30.550 INFO:teuthology.orchestra.run.smithi082.stdout:(10/174): ceph-common-19.0.0-1681.gaba3c1d2.el9 7.8 MB/s | 20 MB 00:02 2024-03-16T17:54:30.625 INFO:teuthology.orchestra.run.smithi082.stdout:(11/174): cephfs-mirror-19.0.0-1681.gaba3c1d2.e 2.9 MB/s | 218 kB 00:00 2024-03-16T17:54:30.701 INFO:teuthology.orchestra.run.smithi082.stdout:(12/174): cephfs-shell-19.0.0-1681.gaba3c1d2.el 365 kB/s | 27 kB 00:00 2024-03-16T17:54:30.801 INFO:teuthology.orchestra.run.smithi082.stdout:(13/174): libcephfs-devel-19.0.0-1681.gaba3c1d2 315 kB/s | 31 kB 00:00 2024-03-16T17:54:30.977 INFO:teuthology.orchestra.run.smithi082.stdout:(14/174): libcephfs2-19.0.0-1681.gaba3c1d2.el9. 4.2 MB/s | 744 kB 00:00 2024-03-16T17:54:31.068 INFO:teuthology.orchestra.run.smithi082.stdout:(15/174): libcephsqlite-19.0.0-1681.gaba3c1d2.e 1.8 MB/s | 163 kB 00:00 2024-03-16T17:54:31.176 INFO:teuthology.orchestra.run.smithi082.stdout:(16/174): librados-devel-19.0.0-1681.gaba3c1d2. 1.2 MB/s | 127 kB 00:00 2024-03-16T17:54:31.302 INFO:teuthology.orchestra.run.smithi082.stdout:(17/174): libradosstriper1-19.0.0-1681.gaba3c1d 3.9 MB/s | 498 kB 00:00 2024-03-16T17:54:31.877 INFO:teuthology.orchestra.run.smithi082.stdout:(18/174): ceph-osd-19.0.0-1681.gaba3c1d2.el9.x8 6.4 MB/s | 17 MB 00:02 2024-03-16T17:54:32.028 INFO:teuthology.orchestra.run.smithi082.stdout:(19/174): python3-ceph-argparse-19.0.0-1681.gab 303 kB/s | 45 kB 00:00 2024-03-16T17:54:32.128 INFO:teuthology.orchestra.run.smithi082.stdout:(20/174): librgw2-19.0.0-1681.gaba3c1d2.el9.x86 6.2 MB/s | 5.1 MB 00:00 2024-03-16T17:54:32.161 INFO:teuthology.orchestra.run.smithi082.stdout:(21/174): python3-ceph-common-19.0.0-1681.gaba3 990 kB/s | 131 kB 00:00 2024-03-16T17:54:32.186 INFO:teuthology.orchestra.run.smithi082.stdout:(22/174): python3-cephfs-19.0.0-1681.gaba3c1d2. 2.7 MB/s | 161 kB 00:00 2024-03-16T17:54:32.237 INFO:teuthology.orchestra.run.smithi082.stdout:(23/174): python3-rados-19.0.0-1681.gaba3c1d2.e 4.2 MB/s | 321 kB 00:00 2024-03-16T17:54:32.271 INFO:teuthology.orchestra.run.smithi082.stdout:(24/174): python3-rbd-19.0.0-1681.gaba3c1d2.el9 3.5 MB/s | 298 kB 00:00 2024-03-16T17:54:32.313 INFO:teuthology.orchestra.run.smithi082.stdout:(25/174): python3-rgw-19.0.0-1681.gaba3c1d2.el9 1.3 MB/s | 99 kB 00:00 2024-03-16T17:54:32.338 INFO:teuthology.orchestra.run.smithi082.stdout:(26/174): rbd-fuse-19.0.0-1681.gaba3c1d2.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-16T17:54:32.405 INFO:teuthology.orchestra.run.smithi082.stdout:(27/174): rbd-nbd-19.0.0-1681.gaba3c1d2.el9.x86 2.5 MB/s | 171 kB 00:00 2024-03-16T17:54:32.488 INFO:teuthology.orchestra.run.smithi082.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1681.g 381 kB/s | 31 kB 00:00 2024-03-16T17:54:32.580 INFO:teuthology.orchestra.run.smithi082.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1681.gaba3c1d 1.5 MB/s | 138 kB 00:00 2024-03-16T17:54:32.672 INFO:teuthology.orchestra.run.smithi082.stdout:(30/174): rbd-mirror-19.0.0-1681.gaba3c1d2.el9. 8.6 MB/s | 3.1 MB 00:00 2024-03-16T17:54:32.889 INFO:teuthology.orchestra.run.smithi082.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1681.gaba3c 5.8 MB/s | 1.8 MB 00:00 2024-03-16T17:54:32.989 INFO:teuthology.orchestra.run.smithi082.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1681.gab 2.4 MB/s | 248 kB 00:00 2024-03-16T17:54:33.082 INFO:teuthology.orchestra.run.smithi082.stdout:(33/174): ceph-mgr-rook-19.0.0-1681.gaba3c1d2.e 544 kB/s | 49 kB 00:00 2024-03-16T17:54:33.181 INFO:teuthology.orchestra.run.smithi082.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1681.ga 155 kB/s | 15 kB 00:00 2024-03-16T17:54:33.282 INFO:teuthology.orchestra.run.smithi082.stdout:(35/174): ceph-volume-19.0.0-1681.gaba3c1d2.el9 2.6 MB/s | 264 kB 00:00 2024-03-16T17:54:33.490 INFO:teuthology.orchestra.run.smithi082.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 9.1 MB/s | 7.4 MB 00:00 2024-03-16T17:54:33.532 INFO:teuthology.orchestra.run.smithi082.stdout:(37/174): cephadm-19.0.0-1681.gaba3c1d2.el9.noa 2.9 MB/s | 748 kB 00:00 2024-03-16T17:54:33.557 INFO:teuthology.orchestra.run.smithi082.stdout:(38/174): cephfs-top-19.0.0-1681.gaba3c1d2.el9. 276 kB/s | 18 kB 00:00 2024-03-16T17:54:34.208 INFO:teuthology.orchestra.run.smithi082.stdout:(39/174): ceph-test-19.0.0-1681.gaba3c1d2.el9.x 12 MB/s | 47 MB 00:03 2024-03-16T17:54:34.266 INFO:teuthology.orchestra.run.smithi082.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 904 kB/s | 662 kB 00:00 2024-03-16T17:54:34.292 INFO:teuthology.orchestra.run.smithi082.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 106 kB 00:00 2024-03-16T17:54:34.333 INFO:teuthology.orchestra.run.smithi082.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 4.7 MB/s | 307 kB 00:00 2024-03-16T17:54:34.377 INFO:teuthology.orchestra.run.smithi082.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 389 kB/s | 32 kB 00:00 2024-03-16T17:54:34.426 INFO:teuthology.orchestra.run.smithi082.stdout:(44/174): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 920 kB 00:00 2024-03-16T17:54:34.518 INFO:teuthology.orchestra.run.smithi082.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 131 kB/s | 17 kB 00:00 2024-03-16T17:54:34.593 INFO:teuthology.orchestra.run.smithi082.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-03-16T17:54:34.618 INFO:teuthology.orchestra.run.smithi082.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 54 kB/s | 10 kB 00:00 2024-03-16T17:54:34.643 INFO:teuthology.orchestra.run.smithi082.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2024-03-16T17:54:34.668 INFO:teuthology.orchestra.run.smithi082.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 923 kB/s | 44 kB 00:00 2024-03-16T17:54:34.710 INFO:teuthology.orchestra.run.smithi082.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.2 MB/s | 661 kB 00:00 2024-03-16T17:54:34.744 INFO:teuthology.orchestra.run.smithi082.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.9 MB/s | 578 kB 00:00 2024-03-16T17:54:34.761 INFO:teuthology.orchestra.run.smithi082.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 540 kB/s | 46 kB 00:00 2024-03-16T17:54:34.794 INFO:teuthology.orchestra.run.smithi082.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 282 kB 00:00 2024-03-16T17:54:34.836 INFO:teuthology.orchestra.run.smithi082.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2024-03-16T17:54:34.861 INFO:teuthology.orchestra.run.smithi082.stdout:(55/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.2 MB/s | 217 kB 00:00 2024-03-16T17:54:34.887 INFO:teuthology.orchestra.run.smithi082.stdout:(56/174): openblas-0.3.21-2.el9.x86_64.rpm 374 kB/s | 34 kB 00:00 2024-03-16T17:54:35.029 INFO:teuthology.orchestra.run.smithi082.stdout:(57/174): postgresql-13.11-1.el9.x86_64.rpm 9.2 MB/s | 1.5 MB 00:00 2024-03-16T17:54:35.129 INFO:teuthology.orchestra.run.smithi082.stdout:(58/174): postgresql-contrib-13.11-1.el9.x86_64 3.3 MB/s | 811 kB 00:00 2024-03-16T17:54:35.221 INFO:teuthology.orchestra.run.smithi082.stdout:(59/174): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-03-16T17:54:35.246 INFO:teuthology.orchestra.run.smithi082.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 666 kB/s | 134 kB 00:00 2024-03-16T17:54:35.388 INFO:teuthology.orchestra.run.smithi082.stdout:(61/174): postgresql-server-13.11-1.el9.x86_64. 22 MB/s | 5.7 MB 00:00 2024-03-16T17:54:35.405 INFO:teuthology.orchestra.run.smithi082.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 538 kB/s | 78 kB 00:00 2024-03-16T17:54:35.455 INFO:teuthology.orchestra.run.smithi082.stdout:(63/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.2 MB/s | 1.0 MB 00:00 2024-03-16T17:54:35.480 INFO:teuthology.orchestra.run.smithi082.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 373 kB/s | 24 kB 00:00 2024-03-16T17:54:35.631 INFO:teuthology.orchestra.run.smithi082.stdout:(65/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-16T17:54:35.656 INFO:teuthology.orchestra.run.smithi082.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 90 kB/s | 18 kB 00:00 2024-03-16T17:54:35.681 INFO:teuthology.orchestra.run.smithi082.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 779 kB/s | 144 kB 00:00 2024-03-16T17:54:35.706 INFO:teuthology.orchestra.run.smithi082.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 432 kB/s | 32 kB 00:00 2024-03-16T17:54:35.740 INFO:teuthology.orchestra.run.smithi082.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 217 kB 00:00 2024-03-16T17:54:35.865 INFO:teuthology.orchestra.run.smithi082.stdout:(70/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-03-16T17:54:35.882 INFO:teuthology.orchestra.run.smithi082.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-16T17:54:35.907 INFO:teuthology.orchestra.run.smithi082.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 451 kB/s | 69 kB 00:00 2024-03-16T17:54:35.949 INFO:teuthology.orchestra.run.smithi082.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2024-03-16T17:54:35.974 INFO:teuthology.orchestra.run.smithi082.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-03-16T17:54:36.008 INFO:teuthology.orchestra.run.smithi082.stdout:(75/174): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 238 kB 00:00 2024-03-16T17:54:36.033 INFO:teuthology.orchestra.run.smithi082.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-03-16T17:54:36.058 INFO:teuthology.orchestra.run.smithi082.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 596 kB/s | 47 kB 00:00 2024-03-16T17:54:36.101 INFO:teuthology.orchestra.run.smithi082.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 654 kB/s | 43 kB 00:00 2024-03-16T17:54:36.251 INFO:teuthology.orchestra.run.smithi082.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 202 kB/s | 39 kB 00:00 2024-03-16T17:54:36.434 INFO:teuthology.orchestra.run.smithi082.stdout:(80/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-03-16T17:54:36.459 INFO:teuthology.orchestra.run.smithi082.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 119 kB/s | 41 kB 00:00 2024-03-16T17:54:36.493 INFO:teuthology.orchestra.run.smithi082.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2024-03-16T17:54:36.518 INFO:teuthology.orchestra.run.smithi082.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-03-16T17:54:36.543 INFO:teuthology.orchestra.run.smithi082.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 786 kB/s | 63 kB 00:00 2024-03-16T17:54:36.568 INFO:teuthology.orchestra.run.smithi082.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 599 kB/s | 44 kB 00:00 2024-03-16T17:54:36.602 INFO:teuthology.orchestra.run.smithi082.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-03-16T17:54:36.627 INFO:teuthology.orchestra.run.smithi082.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.9 MB/s | 241 kB 00:00 2024-03-16T17:54:36.677 INFO:teuthology.orchestra.run.smithi082.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-16T17:54:36.703 INFO:teuthology.orchestra.run.smithi082.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 103 kB 00:00 2024-03-16T17:54:36.736 INFO:teuthology.orchestra.run.smithi082.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2024-03-16T17:54:36.770 INFO:teuthology.orchestra.run.smithi082.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 187 kB 00:00 2024-03-16T17:54:36.904 INFO:teuthology.orchestra.run.smithi082.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-03-16T17:54:36.929 INFO:teuthology.orchestra.run.smithi082.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-03-16T17:54:36.962 INFO:teuthology.orchestra.run.smithi082.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-03-16T17:54:37.004 INFO:teuthology.orchestra.run.smithi082.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-03-16T17:54:37.188 INFO:teuthology.orchestra.run.smithi082.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 83 kB/s | 21 kB 00:00 2024-03-16T17:54:37.222 INFO:teuthology.orchestra.run.smithi082.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 98 kB/s | 25 kB 00:00 2024-03-16T17:54:37.614 INFO:teuthology.orchestra.run.smithi082.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 578 kB/s | 245 kB 00:00 2024-03-16T17:54:37.647 INFO:teuthology.orchestra.run.smithi082.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 584 kB/s | 248 kB 00:00 2024-03-16T17:54:37.764 INFO:teuthology.orchestra.run.smithi082.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 301 kB/s | 35 kB 00:00 2024-03-16T17:54:37.798 INFO:teuthology.orchestra.run.smithi082.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 1.2 MB/s | 216 kB 00:00 2024-03-16T17:54:37.873 INFO:teuthology.orchestra.run.smithi082.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 413 kB/s | 44 kB 00:00 2024-03-16T17:54:37.898 INFO:teuthology.orchestra.run.smithi082.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 942 kB/s | 94 kB 00:00 2024-03-16T17:54:37.956 INFO:teuthology.orchestra.run.smithi082.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 269 kB/s | 22 kB 00:00 2024-03-16T17:54:38.082 INFO:teuthology.orchestra.run.smithi082.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.8 MB/s | 4.1 MB 00:01 2024-03-16T17:54:38.240 INFO:teuthology.orchestra.run.smithi082.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-03-16T17:54:38.474 INFO:teuthology.orchestra.run.smithi082.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 118 kB/s | 19 kB 00:00 2024-03-16T17:54:38.557 INFO:teuthology.orchestra.run.smithi082.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 914 kB/s | 548 kB 00:00 2024-03-16T17:54:38.641 INFO:teuthology.orchestra.run.smithi082.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 628 kB/s | 309 kB 00:00 2024-03-16T17:54:38.683 INFO:teuthology.orchestra.run.smithi082.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 216 kB/s | 26 kB 00:00 2024-03-16T17:54:38.791 INFO:teuthology.orchestra.run.smithi082.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-03-16T17:54:38.817 INFO:teuthology.orchestra.run.smithi082.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 299 kB/s | 49 kB 00:00 2024-03-16T17:54:38.842 INFO:teuthology.orchestra.run.smithi082.stdout:(113/174): libunwind-1.6.2-1.el9.x86_64.rpm 461 kB/s | 67 kB 00:00 2024-03-16T17:54:38.892 INFO:teuthology.orchestra.run.smithi082.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-16T17:54:38.925 INFO:teuthology.orchestra.run.smithi082.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.7 MB/s | 838 kB 00:00 2024-03-16T17:54:38.968 INFO:teuthology.orchestra.run.smithi082.stdout:(116/174): python3-asyncssh-2.13.2-4.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-03-16T17:54:39.001 INFO:teuthology.orchestra.run.smithi082.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 625 kB/s | 43 kB 00:00 2024-03-16T17:54:39.035 INFO:teuthology.orchestra.run.smithi082.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 313 kB/s | 32 kB 00:00 2024-03-16T17:54:39.060 INFO:teuthology.orchestra.run.smithi082.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-03-16T17:54:39.094 INFO:teuthology.orchestra.run.smithi082.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-03-16T17:54:39.127 INFO:teuthology.orchestra.run.smithi082.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-16T17:54:39.186 INFO:teuthology.orchestra.run.smithi082.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 401 kB/s | 35 kB 00:00 2024-03-16T17:54:39.220 INFO:teuthology.orchestra.run.smithi082.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-16T17:54:39.353 INFO:teuthology.orchestra.run.smithi082.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-03-16T17:54:39.370 INFO:teuthology.orchestra.run.smithi082.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 967 kB/s | 144 kB 00:00 2024-03-16T17:54:39.406 INFO:teuthology.orchestra.run.smithi082.stdout:(126/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 888 kB/s | 303 kB 00:00 2024-03-16T17:54:39.421 INFO:teuthology.orchestra.run.smithi082.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 164 kB/s | 11 kB 00:00 2024-03-16T17:54:39.446 INFO:teuthology.orchestra.run.smithi082.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 246 kB/s | 18 kB 00:00 2024-03-16T17:54:39.471 INFO:teuthology.orchestra.run.smithi082.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 349 kB/s | 23 kB 00:00 2024-03-16T17:54:39.496 INFO:teuthology.orchestra.run.smithi082.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-03-16T17:54:39.522 INFO:teuthology.orchestra.run.smithi082.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-03-16T17:54:39.572 INFO:teuthology.orchestra.run.smithi082.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-03-16T17:54:39.597 INFO:teuthology.orchestra.run.smithi082.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 465 kB/s | 46 kB 00:00 2024-03-16T17:54:39.623 INFO:teuthology.orchestra.run.smithi082.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 875 kB/s | 79 kB 00:00 2024-03-16T17:54:39.648 INFO:teuthology.orchestra.run.smithi082.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 777 kB/s | 58 kB 00:00 2024-03-16T17:54:39.681 INFO:teuthology.orchestra.run.smithi082.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-16T17:54:39.706 INFO:teuthology.orchestra.run.smithi082.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 199 kB/s | 16 kB 00:00 2024-03-16T17:54:39.732 INFO:teuthology.orchestra.run.smithi082.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-03-16T17:54:39.774 INFO:teuthology.orchestra.run.smithi082.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 471 kB/s | 31 kB 00:00 2024-03-16T17:54:39.807 INFO:teuthology.orchestra.run.smithi082.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-03-16T17:54:39.832 INFO:teuthology.orchestra.run.smithi082.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-03-16T17:54:39.866 INFO:teuthology.orchestra.run.smithi082.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 617 kB/s | 36 kB 00:00 2024-03-16T17:54:39.891 INFO:teuthology.orchestra.run.smithi082.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 879 kB/s | 51 kB 00:00 2024-03-16T17:54:39.908 INFO:teuthology.orchestra.run.smithi082.stdout:(144/174): python3-pyperclip-1.8.2-2.el9.noarch 125 kB/s | 28 kB 00:00 2024-03-16T17:54:39.942 INFO:teuthology.orchestra.run.smithi082.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-16T17:54:39.967 INFO:teuthology.orchestra.run.smithi082.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-16T17:54:40.000 INFO:teuthology.orchestra.run.smithi082.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-03-16T17:54:40.026 INFO:teuthology.orchestra.run.smithi082.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 242 kB/s | 20 kB 00:00 2024-03-16T17:54:40.060 INFO:teuthology.orchestra.run.smithi082.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-03-16T17:54:40.185 INFO:teuthology.orchestra.run.smithi082.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.7 MB/s | 1.6 MB 00:00 2024-03-16T17:54:40.327 INFO:teuthology.orchestra.run.smithi082.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.6 MB/s | 321 kB 00:00 2024-03-16T17:54:40.544 INFO:teuthology.orchestra.run.smithi082.stdout:(152/174): NetworkManager-1.46.0-1.el9.x86_64.r 11 MB/s | 2.3 MB 00:00 2024-03-16T17:54:40.644 INFO:teuthology.orchestra.run.smithi082.stdout:(153/174): NetworkManager-libnm-1.46.0-1.el9.x8 18 MB/s | 1.8 MB 00:00 2024-03-16T17:54:40.686 INFO:teuthology.orchestra.run.smithi082.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 1.0 MB/s | 42 kB 00:00 2024-03-16T17:54:40.744 INFO:teuthology.orchestra.run.smithi082.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-03-16T17:54:40.787 INFO:teuthology.orchestra.run.smithi082.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-16T17:54:40.828 INFO:teuthology.orchestra.run.smithi082.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 897 kB/s | 36 kB 00:00 2024-03-16T17:54:40.879 INFO:teuthology.orchestra.run.smithi082.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2024-03-16T17:54:40.962 INFO:teuthology.orchestra.run.smithi082.stdout:(159/174): librados2-19.0.0-1681.gaba3c1d2.el9. 3.6 MB/s | 3.4 MB 00:00 2024-03-16T17:54:40.995 INFO:teuthology.orchestra.run.smithi082.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.2 MB/s | 256 kB 00:00 2024-03-16T17:54:41.029 INFO:teuthology.orchestra.run.smithi082.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 347 kB/s | 23 kB 00:00 2024-03-16T17:54:41.063 INFO:teuthology.orchestra.run.smithi082.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-16T17:54:41.113 INFO:teuthology.orchestra.run.smithi082.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 2.7 MB/s | 136 kB 00:00 2024-03-16T17:54:41.138 INFO:teuthology.orchestra.run.smithi082.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-03-16T17:54:41.164 INFO:teuthology.orchestra.run.smithi082.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.3 MB/s | 64 kB 00:00 2024-03-16T17:54:41.189 INFO:teuthology.orchestra.run.smithi082.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 583 kB/s | 29 kB 00:00 2024-03-16T17:54:41.248 INFO:teuthology.orchestra.run.smithi082.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 15 MB/s | 1.2 MB 00:00 2024-03-16T17:54:41.349 INFO:teuthology.orchestra.run.smithi082.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-16T17:54:41.424 INFO:teuthology.orchestra.run.smithi082.stdout:(169/174): util-linux-2.37.4-18.el9.x86_64.rpm 13 MB/s | 2.3 MB 00:00 2024-03-16T17:54:41.508 INFO:teuthology.orchestra.run.smithi082.stdout:(170/174): librbd1-19.0.0-1681.gaba3c1d2.el9.x8 2.3 MB/s | 3.1 MB 00:01 2024-03-16T17:54:41.534 INFO:teuthology.orchestra.run.smithi082.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 287 kB/s | 31 kB 00:00 2024-03-16T17:54:41.567 INFO:teuthology.orchestra.run.smithi082.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 2.1 MB/s | 465 kB 00:00 2024-03-16T17:54:41.701 INFO:teuthology.orchestra.run.smithi082.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.6 MB/s | 1.1 MB 00:00 2024-03-16T17:54:41.801 INFO:teuthology.orchestra.run.smithi082.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 279 kB/s | 74 kB 00:00 2024-03-16T17:54:41.813 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-03-16T17:54:41.813 INFO:teuthology.orchestra.run.smithi082.stdout:Total 16 MB/s | 224 MB 00:14 2024-03-16T17:54:43.240 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-03-16T17:54:43.615 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-03-16T17:54:43.615 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-03-16T17:54:45.860 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-03-16T17:54:45.865 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-03-16T17:54:47.797 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-03-16T17:54:53.086 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-03-16T17:54:53.257 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T17:54:53.306 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T17:54:53.352 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T17:54:53.395 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T17:54:53.499 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T17:54:53.575 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-16T17:54:53.641 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T17:54:53.688 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T17:54:53.717 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T17:54:53.806 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T17:54:54.220 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T17:54:54.290 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-16T17:54:54.456 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-16T17:54:54.496 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-16T17:54:54.629 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-16T17:54:54.675 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 12/198 2024-03-16T17:54:54.733 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T17:54:54.996 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T17:54:55.046 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T17:54:55.072 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-16T17:54:55.201 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-16T17:54:55.239 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 17/198 2024-03-16T17:54:55.310 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 17/198 2024-03-16T17:54:55.356 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T17:54:55.383 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T17:54:55.440 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T17:54:55.706 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T17:54:55.749 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T17:54:55.800 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T17:54:56.154 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T17:54:56.334 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T17:54:56.404 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T17:54:56.405 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-16T17:54:56.406 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T17:54:56.426 INFO:teuthology.orchestra.run.smithi090.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-16T17:54:56.428 INFO:teuthology.orchestra.run.smithi090.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-16T17:54:56.429 INFO:teuthology.orchestra.run.smithi090.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-16T17:54:56.430 INFO:teuthology.orchestra.run.smithi090.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-16T17:54:56.433 INFO:teuthology.orchestra.run.smithi090.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T17:54:56.436 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-16T17:54:56.438 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-16T17:54:56.529 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-03-16T17:54:56.551 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================================ 2024-03-16T17:54:56.551 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-03-16T17:54:56.551 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================================ 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 6.6 k 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 5.3 M 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 887 k 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 145 k 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 1.5 M 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 138 k 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 1.8 M 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 7.4 M 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 49 k 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 9.8 M 2024-03-16T17:54:56.552 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 47 M 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 264 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 748 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 218 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 27 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 18 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 31 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 744 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 127 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-16T17:54:56.553 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 161 k 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 321 k 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 298 k 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 99 k 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 85 k 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.1 M 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 171 k 2024-03-16T17:54:56.554 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-16T17:54:56.555 INFO:teuthology.orchestra.run.smithi090.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.4 M 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 3.1 M 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-16T17:54:56.556 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 20 M 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 31 k 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 2.3 M 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 248 k 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 4.7 M 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 17 M 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1681.gaba3c1d2.el9 ceph-noarch 15 k 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 25 k 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-16T17:54:56.557 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 163 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-16T17:54:56.558 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 498 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 5.1 M 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-16T17:54:56.559 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 45 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.0.0-1681.gaba3c1d2.el9 ceph 131 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-16T17:54:56.560 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-16T17:54:56.561 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-16T17:54:56.562 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-16T17:54:56.563 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-16T17:54:56.564 INFO:teuthology.orchestra.run.smithi090.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-16T17:54:56.565 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-16T17:54:56.565 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-16T17:54:56.565 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-03-16T17:54:56.565 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-16T17:54:56.565 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:54:56.565 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-03-16T17:54:56.565 INFO:teuthology.orchestra.run.smithi090.stdout:============================================================================================ 2024-03-16T17:54:56.565 INFO:teuthology.orchestra.run.smithi090.stdout:Install 150 Packages 2024-03-16T17:54:56.565 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 24 Packages 2024-03-16T17:54:56.565 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:54:56.566 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 224 M 2024-03-16T17:54:56.566 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-03-16T17:54:56.617 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T17:54:56.670 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T17:54:57.004 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T17:54:57.190 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T17:54:57.309 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T17:54:57.359 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T17:54:57.429 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T17:54:57.445 INFO:teuthology.orchestra.run.smithi090.stdout:(1/174): ceph-19.0.0-1681.gaba3c1d2.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-03-16T17:54:57.506 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T17:54:57.621 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T17:54:57.661 INFO:teuthology.orchestra.run.smithi090.stdout:(2/174): ceph-fuse-19.0.0-1681.gaba3c1d2.el9.x8 4.0 MB/s | 887 kB 00:00 2024-03-16T17:54:57.710 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T17:54:57.729 INFO:teuthology.orchestra.run.smithi090.stdout:(3/174): ceph-immutable-object-cache-19.0.0-168 2.1 MB/s | 145 kB 00:00 2024-03-16T17:54:57.760 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T17:54:57.850 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T17:54:57.890 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T17:54:57.922 INFO:teuthology.orchestra.run.smithi090.stdout:(4/174): ceph-base-19.0.0-1681.gaba3c1d2.el9.x8 7.5 MB/s | 5.3 MB 00:00 2024-03-16T17:54:57.950 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T17:54:57.993 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T17:54:58.026 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T17:54:58.030 INFO:teuthology.orchestra.run.smithi090.stdout:(5/174): ceph-mds-19.0.0-1681.gaba3c1d2.el9.x86 7.7 MB/s | 2.3 MB 00:00 2024-03-16T17:54:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:(6/174): ceph-mgr-19.0.0-1681.gaba3c1d2.el9.x86 8.2 MB/s | 1.5 MB 00:00 2024-03-16T17:54:58.138 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T17:54:58.171 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9. 39/198 2024-03-16T17:54:58.224 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el 40/198 2024-03-16T17:54:58.262 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_6 41/198 2024-03-16T17:54:58.380 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T17:54:58.439 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T17:54:58.536 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T17:54:58.681 INFO:teuthology.orchestra.run.smithi090.stdout:(7/174): ceph-mon-19.0.0-1681.gaba3c1d2.el9.x86 7.2 MB/s | 4.7 MB 00:00 2024-03-16T17:54:58.709 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T17:54:58.776 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T17:54:59.459 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T17:54:59.548 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T17:54:59.632 INFO:teuthology.orchestra.run.smithi090.stdout:(8/174): ceph-osd-19.0.0-1681.gaba3c1d2.el9.x86 11 MB/s | 17 MB 00:01 2024-03-16T17:54:59.670 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T17:54:59.682 INFO:teuthology.orchestra.run.smithi090.stdout:(9/174): ceph-selinux-19.0.0-1681.gaba3c1d2.el9 514 kB/s | 25 kB 00:00 2024-03-16T17:54:59.725 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T17:54:59.750 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T17:54:59.799 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T17:54:59.867 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T17:54:59.902 INFO:teuthology.orchestra.run.smithi090.stdout:(10/174): ceph-radosgw-19.0.0-1681.gaba3c1d2.el 8.0 MB/s | 9.8 MB 00:01 2024-03-16T17:54:59.917 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_6 53/198 2024-03-16T17:54:59.957 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T17:55:00.339 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T17:55:00.557 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T17:55:00.637 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T17:55:00.717 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T17:55:00.866 INFO:teuthology.orchestra.run.smithi090.stdout:(11/174): cephfs-mirror-19.0.0-1681.gaba3c1d2.e 227 kB/s | 218 kB 00:00 2024-03-16T17:55:00.922 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T17:55:00.966 INFO:teuthology.orchestra.run.smithi090.stdout:(12/174): cephfs-shell-19.0.0-1681.gaba3c1d2.el 273 kB/s | 27 kB 00:00 2024-03-16T17:55:00.976 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T17:55:01.051 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T17:55:01.066 INFO:teuthology.orchestra.run.smithi090.stdout:(13/174): libcephfs-devel-19.0.0-1681.gaba3c1d2 315 kB/s | 31 kB 00:00 2024-03-16T17:55:01.146 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T17:55:01.192 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T17:55:01.232 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T17:55:01.321 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T17:55:01.367 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T17:55:01.429 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T17:55:01.459 INFO:teuthology.orchestra.run.smithi090.stdout:(14/174): libcephfs2-19.0.0-1681.gaba3c1d2.el9. 1.9 MB/s | 744 kB 00:00 2024-03-16T17:55:01.494 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T17:55:01.601 INFO:teuthology.orchestra.run.smithi090.stdout:(15/174): libcephsqlite-19.0.0-1681.gaba3c1d2.e 1.1 MB/s | 163 kB 00:00 2024-03-16T17:55:01.684 INFO:teuthology.orchestra.run.smithi090.stdout:(16/174): librados-devel-19.0.0-1681.gaba3c1d2. 1.5 MB/s | 127 kB 00:00 2024-03-16T17:55:01.684 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T17:55:01.746 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T17:55:01.785 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T17:55:01.852 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T17:55:01.960 INFO:teuthology.orchestra.run.smithi090.stdout:(17/174): libradosstriper1-19.0.0-1681.gaba3c1d 1.8 MB/s | 498 kB 00:00 2024-03-16T17:55:01.991 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T17:55:02.032 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T17:55:02.079 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T17:55:02.115 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T17:55:02.149 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T17:55:02.360 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T17:55:02.534 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T17:55:02.681 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T17:55:02.743 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T17:55:02.806 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T17:55:02.880 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T17:55:03.123 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T17:55:03.198 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T17:55:03.380 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T17:55:03.427 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T17:55:03.468 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T17:55:03.514 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T17:55:03.556 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T17:55:03.602 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T17:55:03.669 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T17:55:03.728 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T17:55:03.768 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T17:55:03.827 INFO:teuthology.orchestra.run.smithi090.stdout:(18/174): librgw2-19.0.0-1681.gaba3c1d2.el9.x86 2.7 MB/s | 5.1 MB 00:01 2024-03-16T17:55:03.885 INFO:teuthology.orchestra.run.smithi090.stdout:(19/174): python3-ceph-argparse-19.0.0-1681.gab 788 kB/s | 45 kB 00:00 2024-03-16T17:55:03.892 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T17:55:03.942 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T17:55:03.952 INFO:teuthology.orchestra.run.smithi090.stdout:(20/174): python3-ceph-common-19.0.0-1681.gaba3 1.9 MB/s | 131 kB 00:00 2024-03-16T17:55:03.974 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T17:55:04.028 INFO:teuthology.orchestra.run.smithi090.stdout:(21/174): python3-cephfs-19.0.0-1681.gaba3c1d2. 2.1 MB/s | 161 kB 00:00 2024-03-16T17:55:04.047 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T17:55:04.090 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T17:55:04.145 INFO:teuthology.orchestra.run.smithi090.stdout:(22/174): python3-rados-19.0.0-1681.gaba3c1d2.e 2.7 MB/s | 321 kB 00:00 2024-03-16T17:55:04.235 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T17:55:04.279 INFO:teuthology.orchestra.run.smithi090.stdout:(23/174): python3-rbd-19.0.0-1681.gaba3c1d2.el9 2.2 MB/s | 298 kB 00:00 2024-03-16T17:55:04.355 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T17:55:04.395 INFO:teuthology.orchestra.run.smithi090.stdout:(24/174): python3-rgw-19.0.0-1681.gaba3c1d2.el9 857 kB/s | 99 kB 00:00 2024-03-16T17:55:04.418 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T17:55:04.454 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T17:55:04.480 INFO:teuthology.orchestra.run.smithi090.stdout:(25/174): rbd-fuse-19.0.0-1681.gaba3c1d2.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-16T17:55:04.512 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T17:55:04.649 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T17:55:04.720 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T17:55:05.256 INFO:teuthology.orchestra.run.smithi090.stdout:(26/174): ceph-common-19.0.0-1681.gaba3c1d2.el9 2.5 MB/s | 20 MB 00:08 2024-03-16T17:55:05.671 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T17:55:05.728 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T17:55:05.761 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T17:55:05.818 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T17:55:05.890 INFO:teuthology.orchestra.run.smithi090.stdout:(27/174): ceph-test-19.0.0-1681.gaba3c1d2.el9.x 7.6 MB/s | 47 MB 00:06 2024-03-16T17:55:05.923 INFO:teuthology.orchestra.run.smithi090.stdout:(28/174): rbd-nbd-19.0.0-1681.gaba3c1d2.el9.x86 256 kB/s | 171 kB 00:00 2024-03-16T17:55:05.927 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T17:55:05.961 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T17:55:06.015 INFO:teuthology.orchestra.run.smithi090.stdout:(29/174): rbd-mirror-19.0.0-1681.gaba3c1d2.el9. 2.0 MB/s | 3.1 MB 00:01 2024-03-16T17:55:06.041 INFO:teuthology.orchestra.run.smithi090.stdout:(30/174): ceph-grafana-dashboards-19.0.0-1681.g 210 kB/s | 31 kB 00:00 2024-03-16T17:55:06.108 INFO:teuthology.orchestra.run.smithi090.stdout:(31/174): ceph-mgr-cephadm-19.0.0-1681.gaba3c1d 753 kB/s | 138 kB 00:00 2024-03-16T17:55:06.190 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T17:55:06.233 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T17:55:06.283 INFO:teuthology.orchestra.run.smithi090.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1681.gab 1.4 MB/s | 248 kB 00:00 2024-03-16T17:55:06.325 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T17:55:06.392 INFO:teuthology.orchestra.run.smithi090.stdout:(33/174): ceph-mgr-rook-19.0.0-1681.gaba3c1d2.e 458 kB/s | 49 kB 00:00 2024-03-16T17:55:06.500 INFO:teuthology.orchestra.run.smithi090.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1681.ga 142 kB/s | 15 kB 00:00 2024-03-16T17:55:06.609 INFO:teuthology.orchestra.run.smithi090.stdout:(35/174): ceph-mgr-dashboard-19.0.0-1681.gaba3c 3.0 MB/s | 1.8 MB 00:00 2024-03-16T17:55:06.684 INFO:teuthology.orchestra.run.smithi090.stdout:(36/174): ceph-volume-19.0.0-1681.gaba3c1d2.el9 1.4 MB/s | 264 kB 00:00 2024-03-16T17:55:06.768 INFO:teuthology.orchestra.run.smithi090.stdout:(37/174): cephfs-top-19.0.0-1681.gaba3c1d2.el9. 221 kB/s | 18 kB 00:00 2024-03-16T17:55:06.818 INFO:teuthology.orchestra.run.smithi090.stdout:(38/174): cephadm-19.0.0-1681.gaba3c1d2.el9.noa 3.5 MB/s | 748 kB 00:00 2024-03-16T17:55:07.011 INFO:teuthology.orchestra.run.smithi090.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 7.7 MB/s | 7.4 MB 00:00 2024-03-16T17:55:07.105 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T17:55:07.186 INFO:teuthology.orchestra.run.smithi090.stdout:(40/174): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-03-16T17:55:07.211 INFO:teuthology.orchestra.run.smithi090.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 786 kB/s | 106 kB 00:00 2024-03-16T17:55:07.236 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T17:55:07.270 INFO:teuthology.orchestra.run.smithi090.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 3.8 MB/s | 307 kB 00:00 2024-03-16T17:55:07.303 INFO:teuthology.orchestra.run.smithi090.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 345 kB/s | 32 kB 00:00 2024-03-16T17:55:07.345 INFO:teuthology.orchestra.run.smithi090.stdout:(44/174): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 920 kB 00:00 2024-03-16T17:55:07.396 INFO:teuthology.orchestra.run.smithi090.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 189 kB/s | 17 kB 00:00 2024-03-16T17:55:07.488 INFO:teuthology.orchestra.run.smithi090.stdout:(46/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-03-16T17:55:07.513 INFO:teuthology.orchestra.run.smithi090.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 62 kB/s | 10 kB 00:00 2024-03-16T17:55:07.538 INFO:teuthology.orchestra.run.smithi090.stdout:(48/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2024-03-16T17:55:07.573 INFO:teuthology.orchestra.run.smithi090.stdout:(49/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 767 kB/s | 44 kB 00:00 2024-03-16T17:55:07.607 INFO:teuthology.orchestra.run.smithi090.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2024-03-16T17:55:07.649 INFO:teuthology.orchestra.run.smithi090.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 5.4 MB/s | 578 kB 00:00 2024-03-16T17:55:07.674 INFO:teuthology.orchestra.run.smithi090.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 755 kB/s | 46 kB 00:00 2024-03-16T17:55:07.699 INFO:teuthology.orchestra.run.smithi090.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.2 MB/s | 282 kB 00:00 2024-03-16T17:55:07.733 INFO:teuthology.orchestra.run.smithi090.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 187 kB 00:00 2024-03-16T17:55:07.775 INFO:teuthology.orchestra.run.smithi090.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 470 kB/s | 34 kB 00:00 2024-03-16T17:55:07.817 INFO:teuthology.orchestra.run.smithi090.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.8 MB/s | 217 kB 00:00 2024-03-16T17:55:07.959 INFO:teuthology.orchestra.run.smithi090.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-03-16T17:55:08.026 INFO:teuthology.orchestra.run.smithi090.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 6.5 MB/s | 1.5 MB 00:00 2024-03-16T17:55:08.068 INFO:teuthology.orchestra.run.smithi090.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.4 MB/s | 811 kB 00:00 2024-03-16T17:55:08.102 INFO:teuthology.orchestra.run.smithi090.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 994 kB/s | 134 kB 00:00 2024-03-16T17:55:08.169 INFO:teuthology.orchestra.run.smithi090.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-03-16T17:55:08.319 INFO:teuthology.orchestra.run.smithi090.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 23 MB/s | 5.7 MB 00:00 2024-03-16T17:55:08.353 INFO:teuthology.orchestra.run.smithi090.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 314 kB/s | 78 kB 00:00 2024-03-16T17:55:08.503 INFO:teuthology.orchestra.run.smithi090.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2024-03-16T17:55:08.528 INFO:teuthology.orchestra.run.smithi090.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 151 kB/s | 24 kB 00:00 2024-03-16T17:55:08.553 INFO:teuthology.orchestra.run.smithi090.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 97 kB/s | 18 kB 00:00 2024-03-16T17:55:08.595 INFO:teuthology.orchestra.run.smithi090.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 786 kB/s | 32 kB 00:00 2024-03-16T17:55:08.637 INFO:teuthology.orchestra.run.smithi090.stdout:(68/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2024-03-16T17:55:08.737 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T17:55:08.763 INFO:teuthology.orchestra.run.smithi090.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2024-03-16T17:55:08.796 INFO:teuthology.orchestra.run.smithi090.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-03-16T17:55:08.821 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T17:55:08.822 INFO:teuthology.orchestra.run.smithi090.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-16T17:55:08.855 INFO:teuthology.orchestra.run.smithi090.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 69 kB 00:00 2024-03-16T17:55:08.860 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T17:55:08.880 INFO:teuthology.orchestra.run.smithi090.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 501 kB/s | 41 kB 00:00 2024-03-16T17:55:08.890 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T17:55:08.914 INFO:teuthology.orchestra.run.smithi090.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 238 kB 00:00 2024-03-16T17:55:08.931 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T17:55:08.948 INFO:teuthology.orchestra.run.smithi090.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2024-03-16T17:55:08.974 INFO:teuthology.orchestra.run.smithi090.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2024-03-16T17:55:09.000 INFO:teuthology.orchestra.run.smithi090.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 578 kB/s | 47 kB 00:00 2024-03-16T17:55:09.058 INFO:teuthology.orchestra.run.smithi090.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 519 kB/s | 43 kB 00:00 2024-03-16T17:55:09.303 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T17:55:09.337 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 124/198 2024-03-16T17:55:09.457 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 124/198 2024-03-16T17:55:09.485 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 125/198 2024-03-16T17:55:09.492 INFO:teuthology.orchestra.run.smithi090.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-03-16T17:55:09.517 INFO:teuthology.orchestra.run.smithi090.stdout:(80/174): python3-toml-0.10.2-6.el9.noarch.rpm 77 kB/s | 39 kB 00:00 2024-03-16T17:55:09.540 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T17:55:09.542 INFO:teuthology.orchestra.run.smithi090.stdout:(81/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 87 kB/s | 41 kB 00:00 2024-03-16T17:55:09.584 INFO:teuthology.orchestra.run.smithi090.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 56 kB 00:00 2024-03-16T17:55:09.618 INFO:teuthology.orchestra.run.smithi090.stdout:(83/174): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 300 kB 00:00 2024-03-16T17:55:09.643 INFO:teuthology.orchestra.run.smithi090.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 634 kB/s | 63 kB 00:00 2024-03-16T17:55:09.702 INFO:teuthology.orchestra.run.smithi090.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 387 kB/s | 44 kB 00:00 2024-03-16T17:55:09.736 INFO:teuthology.orchestra.run.smithi090.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 868 kB/s | 79 kB 00:00 2024-03-16T17:55:09.761 INFO:teuthology.orchestra.run.smithi090.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 241 kB 00:00 2024-03-16T17:55:09.820 INFO:teuthology.orchestra.run.smithi090.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.1 MB 00:00 2024-03-16T17:55:09.845 INFO:teuthology.orchestra.run.smithi090.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 1.7 MB/s | 124 kB 00:00 2024-03-16T17:55:09.870 INFO:teuthology.orchestra.run.smithi090.stdout:(90/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-16T17:55:09.904 INFO:teuthology.orchestra.run.smithi090.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2024-03-16T17:55:10.042 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T17:55:10.082 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 128/198 2024-03-16T17:55:10.172 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 128/198 2024-03-16T17:55:10.201 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 129/198 2024-03-16T17:55:10.338 INFO:teuthology.orchestra.run.smithi090.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 452 kB/s | 195 kB 00:00 2024-03-16T17:55:10.371 INFO:teuthology.orchestra.run.smithi090.stdout:(93/174): jq-1.6-16.el9.x86_64.rpm 357 kB/s | 187 kB 00:00 2024-03-16T17:55:10.438 INFO:teuthology.orchestra.run.smithi090.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-03-16T17:55:10.464 INFO:teuthology.orchestra.run.smithi090.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 235 kB/s | 21 kB 00:00 2024-03-16T17:55:10.498 INFO:teuthology.orchestra.run.smithi090.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 438 kB/s | 25 kB 00:00 2024-03-16T17:55:10.548 INFO:teuthology.orchestra.run.smithi090.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-03-16T17:55:10.598 INFO:teuthology.orchestra.run.smithi090.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-03-16T17:55:10.649 INFO:teuthology.orchestra.run.smithi090.stdout:(99/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.3 MB/s | 216 kB 00:00 2024-03-16T17:55:10.699 INFO:teuthology.orchestra.run.smithi090.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 713 kB/s | 35 kB 00:00 2024-03-16T17:55:10.733 INFO:teuthology.orchestra.run.smithi090.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.3 MB/s | 248 kB 00:00 2024-03-16T17:55:10.767 INFO:teuthology.orchestra.run.smithi090.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 659 kB/s | 44 kB 00:00 2024-03-16T17:55:10.792 INFO:teuthology.orchestra.run.smithi090.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.5 MB/s | 94 kB 00:00 2024-03-16T17:55:10.843 INFO:teuthology.orchestra.run.smithi090.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 300 kB/s | 22 kB 00:00 2024-03-16T17:55:11.001 INFO:teuthology.orchestra.run.smithi090.stdout:(105/174): openssl-devel-3.0.7-27.el9.x86_64.rp 7.7 MB/s | 4.1 MB 00:00 2024-03-16T17:55:11.136 INFO:teuthology.orchestra.run.smithi090.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-03-16T17:55:11.494 INFO:teuthology.orchestra.run.smithi090.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 67 kB/s | 19 kB 00:00 2024-03-16T17:55:11.536 INFO:teuthology.orchestra.run.smithi090.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 792 kB/s | 548 kB 00:00 2024-03-16T17:55:11.620 INFO:teuthology.orchestra.run.smithi090.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 314 kB/s | 26 kB 00:00 2024-03-16T17:55:11.653 INFO:teuthology.orchestra.run.smithi090.stdout:(110/174): gperftools-libs-2.9.1-2.el9.x86_64.r 531 kB/s | 309 kB 00:00 2024-03-16T17:55:11.705 INFO:teuthology.orchestra.run.smithi090.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 580 kB/s | 49 kB 00:00 2024-03-16T17:55:11.738 INFO:teuthology.orchestra.run.smithi090.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 804 kB/s | 67 kB 00:00 2024-03-16T17:55:11.789 INFO:teuthology.orchestra.run.smithi090.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-03-16T17:55:11.915 INFO:teuthology.orchestra.run.smithi090.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-03-16T17:55:11.965 INFO:teuthology.orchestra.run.smithi090.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-03-16T17:55:11.990 INFO:teuthology.orchestra.run.smithi090.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 583 kB/s | 43 kB 00:00 2024-03-16T17:55:12.041 INFO:teuthology.orchestra.run.smithi090.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 431 kB/s | 32 kB 00:00 2024-03-16T17:55:12.074 INFO:teuthology.orchestra.run.smithi090.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2024-03-16T17:55:12.191 INFO:teuthology.orchestra.run.smithi090.stdout:(119/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-03-16T17:55:12.262 INFO:teuthology.orchestra.run.smithi090.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 778 kB/s | 172 kB 00:00 2024-03-16T17:55:12.300 INFO:teuthology.orchestra.run.smithi090.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 1.6 MB/s | 358 kB 00:00 2024-03-16T17:55:12.334 INFO:teuthology.orchestra.run.smithi090.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-03-16T17:55:12.368 INFO:teuthology.orchestra.run.smithi090.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 338 kB/s | 35 kB 00:00 2024-03-16T17:55:12.401 INFO:teuthology.orchestra.run.smithi090.stdout:(124/174): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-03-16T17:55:12.443 INFO:teuthology.orchestra.run.smithi090.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-03-16T17:55:12.477 INFO:teuthology.orchestra.run.smithi090.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 143 kB/s | 11 kB 00:00 2024-03-16T17:55:12.553 INFO:teuthology.orchestra.run.smithi090.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 9.4 MB/s | 2.0 MB 00:00 2024-03-16T17:55:12.578 INFO:teuthology.orchestra.run.smithi090.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2024-03-16T17:55:12.603 INFO:teuthology.orchestra.run.smithi090.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2024-03-16T17:55:12.629 INFO:teuthology.orchestra.run.smithi090.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2024-03-16T17:55:12.680 INFO:teuthology.orchestra.run.smithi090.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 260 kB/s | 20 kB 00:00 2024-03-16T17:55:12.705 INFO:teuthology.orchestra.run.smithi090.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 608 kB/s | 46 kB 00:00 2024-03-16T17:55:12.764 INFO:teuthology.orchestra.run.smithi090.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-03-16T17:55:12.789 INFO:teuthology.orchestra.run.smithi090.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 695 kB/s | 58 kB 00:00 2024-03-16T17:55:12.815 INFO:teuthology.orchestra.run.smithi090.stdout:(135/174): python3-more-itertools-8.12.0-2.el9. 590 kB/s | 79 kB 00:00 2024-03-16T17:55:12.816 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-16T17:55:12.848 INFO:teuthology.orchestra.run.smithi090.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-16T17:55:12.864 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-16T17:55:12.899 INFO:teuthology.orchestra.run.smithi090.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-03-16T17:55:12.924 INFO:teuthology.orchestra.run.smithi090.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 376 kB/s | 28 kB 00:00 2024-03-16T17:55:12.949 INFO:teuthology.orchestra.run.smithi090.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 673 kB/s | 90 kB 00:00 2024-03-16T17:55:12.975 INFO:teuthology.orchestra.run.smithi090.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 411 kB/s | 31 kB 00:00 2024-03-16T17:55:13.008 INFO:teuthology.orchestra.run.smithi090.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-03-16T17:55:13.034 INFO:teuthology.orchestra.run.smithi090.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 709 kB/s | 59 kB 00:00 2024-03-16T17:55:13.059 INFO:teuthology.orchestra.run.smithi090.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 429 kB/s | 36 kB 00:00 2024-03-16T17:55:13.084 INFO:teuthology.orchestra.run.smithi090.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 681 kB/s | 51 kB 00:00 2024-03-16T17:55:13.118 INFO:teuthology.orchestra.run.smithi090.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-03-16T17:55:13.143 INFO:teuthology.orchestra.run.smithi090.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-03-16T17:55:13.186 INFO:teuthology.orchestra.run.smithi090.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-16T17:55:13.211 INFO:teuthology.orchestra.run.smithi090.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-03-16T17:55:13.245 INFO:teuthology.orchestra.run.smithi090.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-16T17:55:13.378 INFO:teuthology.orchestra.run.smithi090.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-03-16T17:55:13.510 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-16T17:55:13.550 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 131/198 2024-03-16T17:55:13.596 INFO:teuthology.orchestra.run.smithi090.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.0 MB/s | 321 kB 00:00 2024-03-16T17:55:13.713 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 131/198 2024-03-16T17:55:13.713 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T17:55:13.713 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T17:55:13.713 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:13.751 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 132/198 2024-03-16T17:55:13.771 INFO:teuthology.orchestra.run.smithi090.stdout:(152/174): librados2-19.0.0-1681.gaba3c1d2.el9. 6.1 MB/s | 3.4 MB 00:00 2024-03-16T17:55:14.063 INFO:teuthology.orchestra.run.smithi090.stdout:(153/174): NetworkManager-1.46.0-1.el9.x86_64.r 4.9 MB/s | 2.3 MB 00:00 2024-03-16T17:55:14.131 INFO:teuthology.orchestra.run.smithi090.stdout:(154/174): NetworkManager-team-1.46.0-1.el9.x86 633 kB/s | 42 kB 00:00 2024-03-16T17:55:14.223 INFO:teuthology.orchestra.run.smithi090.stdout:(155/174): NetworkManager-tui-1.46.0-1.el9.x86_ 2.7 MB/s | 248 kB 00:00 2024-03-16T17:55:14.282 INFO:teuthology.orchestra.run.smithi090.stdout:(156/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.6 MB/s | 1.8 MB 00:00 2024-03-16T17:55:14.324 INFO:teuthology.orchestra.run.smithi090.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 717 kB/s | 71 kB 00:00 2024-03-16T17:55:14.349 INFO:teuthology.orchestra.run.smithi090.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 551 kB/s | 36 kB 00:00 2024-03-16T17:55:14.408 INFO:teuthology.orchestra.run.smithi090.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.3 MB/s | 192 kB 00:00 2024-03-16T17:55:14.458 INFO:teuthology.orchestra.run.smithi090.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-03-16T17:55:14.483 INFO:teuthology.orchestra.run.smithi090.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 309 kB/s | 23 kB 00:00 2024-03-16T17:55:14.584 INFO:teuthology.orchestra.run.smithi090.stdout:(162/174): librbd1-19.0.0-1681.gaba3c1d2.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-03-16T17:55:14.626 INFO:teuthology.orchestra.run.smithi090.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 648 kB/s | 108 kB 00:00 2024-03-16T17:55:14.660 INFO:teuthology.orchestra.run.smithi090.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 882 kB/s | 155 kB 00:00 2024-03-16T17:55:14.694 INFO:teuthology.orchestra.run.smithi090.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-03-16T17:55:14.719 INFO:teuthology.orchestra.run.smithi090.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 695 kB/s | 64 kB 00:00 2024-03-16T17:55:14.744 INFO:teuthology.orchestra.run.smithi090.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 345 kB/s | 29 kB 00:00 2024-03-16T17:55:14.937 INFO:teuthology.orchestra.run.smithi090.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 5.0 MB/s | 1.2 MB 00:00 2024-03-16T17:55:15.054 INFO:teuthology.orchestra.run.smithi090.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.5 MB/s | 2.2 MB 00:00 2024-03-16T17:55:15.096 INFO:teuthology.orchestra.run.smithi090.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 2.9 MB/s | 465 kB 00:00 2024-03-16T17:55:15.146 INFO:teuthology.orchestra.run.smithi090.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 341 kB/s | 31 kB 00:00 2024-03-16T17:55:15.246 INFO:teuthology.orchestra.run.smithi090.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.6 MB/s | 2.3 MB 00:00 2024-03-16T17:55:15.271 INFO:teuthology.orchestra.run.smithi090.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 597 kB/s | 74 kB 00:00 2024-03-16T17:55:15.331 INFO:teuthology.orchestra.run.smithi090.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.6 MB/s | 1.1 MB 00:00 2024-03-16T17:55:15.350 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-03-16T17:55:15.351 INFO:teuthology.orchestra.run.smithi090.stdout:Total 12 MB/s | 224 MB 00:18 2024-03-16T17:55:16.797 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-03-16T17:55:17.173 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-03-16T17:55:17.173 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-03-16T17:55:19.743 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-03-16T17:55:19.743 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-03-16T17:55:26.333 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 132/198 2024-03-16T17:55:26.333 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2024-03-16T17:55:26.333 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2024-03-16T17:55:26.333 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2024-03-16T17:55:26.333 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2024-03-16T17:55:26.334 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2024-03-16T17:55:26.334 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /root 2024-03-16T17:55:26.334 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2024-03-16T17:55:26.334 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:26.559 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 133/198 2024-03-16T17:55:26.607 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 133/198 2024-03-16T17:55:26.608 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:55:26.608 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T17:55:26.608 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:27.408 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-03-16T17:55:27.565 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-16T17:55:27.614 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-16T17:55:27.652 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T17:55:27.696 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-16T17:55:27.800 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-16T17:55:27.905 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-16T17:55:27.961 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-16T17:55:27.993 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-16T17:55:28.075 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-16T17:55:28.201 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 134/198 2024-03-16T17:55:28.283 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 134/198 2024-03-16T17:55:28.284 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:55:28.284 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T17:55:28.284 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T17:55:28.284 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T17:55:28.284 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:28.483 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-16T17:55:28.518 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 135/198 2024-03-16T17:55:28.551 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-16T17:55:28.587 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 135/198 2024-03-16T17:55:28.587 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:55:28.587 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T17:55:28.587 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T17:55:28.587 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T17:55:28.587 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:28.704 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-16T17:55:28.735 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-16T17:55:28.857 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-16T17:55:28.898 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 12/198 2024-03-16T17:55:28.979 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-16T17:55:29.060 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 136/198 2024-03-16T17:55:29.137 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 136/198 2024-03-16T17:55:29.137 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:55:29.137 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T17:55:29.137 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T17:55:29.138 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T17:55:29.138 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:29.254 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T17:55:29.293 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-16T17:55:29.350 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-16T17:55:29.376 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-16T17:55:29.530 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-16T17:55:29.553 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 17/198 2024-03-16T17:55:29.677 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 17/198 2024-03-16T17:55:29.730 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-16T17:55:29.763 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T17:55:29.826 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-16T17:55:29.926 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 138/198 2024-03-16T17:55:29.955 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 138/198 2024-03-16T17:55:30.003 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.e 139/198 2024-03-16T17:55:30.105 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2. 140/198 2024-03-16T17:55:30.118 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T17:55:30.134 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 141/198 2024-03-16T17:55:30.164 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-16T17:55:30.207 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-16T17:55:30.464 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 141/198 2024-03-16T17:55:30.502 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 142/198 2024-03-16T17:55:30.630 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-16T17:55:30.682 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T17:55:30.780 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-16T17:55:31.026 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T17:55:31.078 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T17:55:31.441 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-16T17:55:31.619 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 142/198 2024-03-16T17:55:31.634 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-16T17:55:31.652 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 143/198 2024-03-16T17:55:31.772 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-16T17:55:31.797 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 143/198 2024-03-16T17:55:31.810 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-16T17:55:31.860 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-16T17:55:31.915 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-16T17:55:31.975 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el 144/198 2024-03-16T17:55:32.004 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 145/198 2024-03-16T17:55:32.021 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-16T17:55:32.075 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 145/198 2024-03-16T17:55:32.075 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:55:32.075 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T17:55:32.075 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T17:55:32.075 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T17:55:32.075 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:32.092 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-16T17:55:32.130 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 146/198 2024-03-16T17:55:32.160 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-16T17:55:32.167 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 146/198 2024-03-16T17:55:32.235 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 147/198 2024-03-16T17:55:32.301 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-16T17:55:32.353 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-16T17:55:32.419 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-16T17:55:32.465 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-16T17:55:32.497 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-16T17:55:32.610 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-16T17:55:32.659 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9. 39/198 2024-03-16T17:55:32.726 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el 40/198 2024-03-16T17:55:32.766 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_6 41/198 2024-03-16T17:55:32.883 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-16T17:55:32.944 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-16T17:55:33.030 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-16T17:55:33.165 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T17:55:33.194 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-16T17:55:33.209 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 149/198 2024-03-16T17:55:33.253 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-16T17:55:33.276 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 149/198 2024-03-16T17:55:33.276 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:55:33.276 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T17:55:33.276 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T17:55:33.276 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T17:55:33.276 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:33.341 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 150/198 2024-03-16T17:55:33.408 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 150/198 2024-03-16T17:55:33.408 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:55:33.408 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T17:55:33.408 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:33.492 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-16T17:55:33.568 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-16T17:55:33.568 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:55:33.568 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T17:55:33.568 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T17:55:33.568 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T17:55:33.568 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:33.892 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-16T17:55:33.926 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 152/198 2024-03-16T17:55:33.988 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-16T17:55:33.996 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 152/198 2024-03-16T17:55:33.996 INFO:teuthology.orchestra.run.smithi082.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:55:33.996 INFO:teuthology.orchestra.run.smithi082.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T17:55:33.996 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T17:55:33.996 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T17:55:33.996 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:34.157 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-16T17:55:34.200 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T17:55:34.230 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-16T17:55:34.300 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-16T17:55:34.377 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-16T17:55:34.430 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_6 53/198 2024-03-16T17:55:34.471 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-16T17:55:34.899 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-16T17:55:35.108 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-16T17:55:35.167 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-16T17:55:35.243 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-16T17:55:35.480 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-16T17:55:35.523 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-16T17:55:35.587 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-16T17:55:35.673 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-16T17:55:35.731 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-16T17:55:35.771 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-16T17:55:35.841 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-16T17:55:35.889 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-16T17:55:35.954 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-16T17:55:36.018 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-16T17:55:36.237 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-16T17:55:36.296 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-16T17:55:36.334 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-16T17:55:36.391 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-16T17:55:36.530 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-16T17:55:36.568 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-16T17:55:36.622 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-16T17:55:36.659 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-16T17:55:36.717 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-16T17:55:36.897 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-16T17:55:37.069 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-16T17:55:37.216 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-16T17:55:37.261 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-16T17:55:37.323 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-16T17:55:37.398 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-16T17:55:37.650 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-16T17:55:37.710 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-16T17:55:37.879 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-16T17:55:37.924 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 87/198 2024-03-16T17:55:37.999 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-16T17:55:38.045 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-16T17:55:38.086 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-16T17:55:38.134 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-16T17:55:38.194 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-16T17:55:38.243 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-16T17:55:38.291 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-16T17:55:38.427 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-16T17:55:38.472 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-16T17:55:38.515 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-16T17:55:38.581 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-16T17:55:38.644 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-16T17:55:38.740 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 153/198 2024-03-16T17:55:38.784 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 154/198 2024-03-16T17:55:38.790 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-16T17:55:38.820 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 155/198 2024-03-16T17:55:38.858 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T17:55:38.905 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-16T17:55:38.966 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-16T17:55:39.011 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-16T17:55:39.070 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-16T17:55:39.122 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 157/198 2024-03-16T17:55:39.208 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-16T17:55:39.241 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T17:55:39.289 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 159/198 2024-03-16T17:55:39.310 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-16T17:55:39.348 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T17:55:39.454 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T17:55:39.512 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T17:55:39.548 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_ 163/198 2024-03-16T17:55:39.592 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T17:55:39.644 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T17:55:39.737 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T17:55:39.849 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T17:55:39.900 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T17:55:40.260 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-16T17:55:40.328 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-16T17:55:40.362 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-16T17:55:40.422 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-16T17:55:40.532 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-16T17:55:40.580 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-16T17:55:40.775 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T17:55:40.812 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-16T17:55:40.822 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T17:55:40.844 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-16T17:55:40.885 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T17:55:40.952 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-16T17:55:41.765 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-16T17:55:41.959 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-16T17:55:42.205 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 170/198 2024-03-16T17:55:42.303 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T17:55:42.390 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T17:55:42.527 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T17:55:42.564 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T17:55:42.597 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T17:55:42.631 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T17:55:42.672 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T17:55:42.673 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T17:55:42.811 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T17:55:42.855 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T17:55:42.855 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T17:55:42.965 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T17:55:43.015 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T17:55:43.064 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T17:55:43.098 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T17:55:43.130 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T17:55:43.197 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T17:55:43.271 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T17:55:43.335 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T17:55:43.368 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T17:55:43.401 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T17:55:43.436 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T17:55:43.469 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T17:55:43.505 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T17:55:43.505 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T17:55:43.531 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-16T17:55:43.639 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-16T17:55:43.683 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-16T17:55:43.754 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-16T17:55:43.793 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-16T17:55:43.818 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T17:55:43.818 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T17:55:43.958 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T17:55:43.989 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T17:55:44.022 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T17:55:44.080 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T17:55:44.123 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T17:55:44.171 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-16T17:55:44.207 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 124/198 2024-03-16T17:55:44.242 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T17:55:44.327 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 124/198 2024-03-16T17:55:44.401 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 125/198 2024-03-16T17:55:44.470 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-16T17:55:44.507 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T17:55:44.507 INFO:teuthology.orchestra.run.smithi082.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-16T17:55:44.507 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:44.984 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-16T17:55:45.044 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 128/198 2024-03-16T17:55:45.137 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 128/198 2024-03-16T17:55:45.172 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 129/198 2024-03-16T17:55:47.813 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-16T17:55:47.854 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-16T17:55:48.527 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 130/198 2024-03-16T17:55:48.567 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 131/198 2024-03-16T17:55:48.745 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 131/198 2024-03-16T17:55:48.745 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-16T17:55:48.745 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-16T17:55:48.745 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:55:48.785 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 132/198 2024-03-16T17:55:49.487 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 1/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 3/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 4/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 5/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 6/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 7/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 8/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 9/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 12/198 2024-03-16T17:55:49.488 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 13/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 14/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_ 15/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 17/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_6 18/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 19/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 20/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el 21/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9. 22/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_6 23/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 24/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 25/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 26/198 2024-03-16T17:55:49.489 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 27/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 28/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 29/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2. 30/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 31/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 32/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 33/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el 34/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 35/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.e 36/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 37/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 38/198 2024-03-16T17:55:49.490 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 39/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T17:55:49.491 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T17:55:49.492 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T17:55:49.493 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T17:55:49.494 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T17:55:49.495 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T17:55:49.496 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T17:55:49.497 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T17:55:49.498 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 153/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T17:55:49.499 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T17:55:49.500 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T17:55:49.501 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T17:55:49.502 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T17:55:49.502 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T17:55:49.502 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T17:55:49.502 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T17:55:51.970 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T17:55:51.971 INFO:teuthology.orchestra.run.smithi082.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T17:55:51.972 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:55:51.973 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T17:55:51.974 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: jq-1.6-16.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.975 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T17:55:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T17:55:51.977 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T17:55:51.978 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T17:55:51.979 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T17:55:51.980 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.981 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T17:55:51.982 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:55:51.983 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-03-16T17:55:52.672 DEBUG:teuthology.parallel:result is None 2024-03-16T17:56:03.125 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 132/198 2024-03-16T17:56:03.125 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-03-16T17:56:03.125 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-03-16T17:56:03.125 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-03-16T17:56:03.125 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-03-16T17:56:03.125 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-03-16T17:56:03.125 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-03-16T17:56:03.125 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-03-16T17:56:03.125 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:03.372 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 133/198 2024-03-16T17:56:03.425 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 133/198 2024-03-16T17:56:03.425 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:56:03.425 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-16T17:56:03.425 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:05.061 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 134/198 2024-03-16T17:56:05.137 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 134/198 2024-03-16T17:56:05.138 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:56:05.138 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-16T17:56:05.138 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T17:56:05.138 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-16T17:56:05.138 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:05.378 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 135/198 2024-03-16T17:56:05.448 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 135/198 2024-03-16T17:56:05.448 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:56:05.448 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-16T17:56:05.448 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T17:56:05.448 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-16T17:56:05.448 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:05.998 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 136/198 2024-03-16T17:56:06.065 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 136/198 2024-03-16T17:56:06.065 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:56:06.066 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-16T17:56:06.066 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T17:56:06.066 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-16T17:56:06.066 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:06.198 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-16T17:56:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 138/198 2024-03-16T17:56:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 138/198 2024-03-16T17:56:06.944 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.e 139/198 2024-03-16T17:56:07.046 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2. 140/198 2024-03-16T17:56:07.080 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 141/198 2024-03-16T17:56:07.407 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 141/198 2024-03-16T17:56:07.440 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 142/198 2024-03-16T17:56:08.555 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 142/198 2024-03-16T17:56:08.582 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 143/198 2024-03-16T17:56:08.724 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 143/198 2024-03-16T17:56:08.896 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el 144/198 2024-03-16T17:56:08.927 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 145/198 2024-03-16T17:56:08.998 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 145/198 2024-03-16T17:56:08.998 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:56:08.998 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-16T17:56:08.998 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T17:56:08.998 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-16T17:56:08.998 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:09.053 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 146/198 2024-03-16T17:56:09.089 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 146/198 2024-03-16T17:56:09.198 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 147/198 2024-03-16T17:56:10.138 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-16T17:56:10.173 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 149/198 2024-03-16T17:56:10.265 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 149/198 2024-03-16T17:56:10.266 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:56:10.266 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-16T17:56:10.266 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T17:56:10.266 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-16T17:56:10.266 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:10.315 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 150/198 2024-03-16T17:56:10.409 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 150/198 2024-03-16T17:56:10.409 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:56:10.409 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-16T17:56:10.409 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:10.500 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-16T17:56:10.578 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-16T17:56:10.578 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:56:10.578 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-16T17:56:10.578 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T17:56:10.578 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-16T17:56:10.578 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:10.968 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 152/198 2024-03-16T17:56:11.037 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 152/198 2024-03-16T17:56:11.038 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-16T17:56:11.038 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-16T17:56:11.038 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T17:56:11.038 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-16T17:56:11.038 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:15.844 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 153/198 2024-03-16T17:56:15.898 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 154/198 2024-03-16T17:56:15.941 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 155/198 2024-03-16T17:56:15.988 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-16T17:56:16.270 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 157/198 2024-03-16T17:56:16.405 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-16T17:56:16.453 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 159/198 2024-03-16T17:56:16.511 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-16T17:56:16.588 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-16T17:56:16.626 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-16T17:56:16.660 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_ 163/198 2024-03-16T17:56:16.697 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-16T17:56:16.748 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-16T17:56:16.841 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-16T17:56:16.976 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-16T17:56:17.006 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-16T17:56:17.949 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T17:56:17.999 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T17:56:18.065 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-16T17:56:19.377 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 170/198 2024-03-16T17:56:19.533 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-16T17:56:19.607 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-16T17:56:19.767 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-16T17:56:19.813 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-16T17:56:19.846 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-16T17:56:19.880 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-16T17:56:19.924 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-16T17:56:19.924 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T17:56:20.075 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-16T17:56:20.126 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-16T17:56:20.127 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T17:56:20.264 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-16T17:56:20.315 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-16T17:56:20.363 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-16T17:56:20.397 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-16T17:56:20.437 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-16T17:56:20.528 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-16T17:56:20.612 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-16T17:56:20.669 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-16T17:56:20.760 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-16T17:56:20.803 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-16T17:56:20.837 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-16T17:56:20.870 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-16T17:56:20.905 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T17:56:20.906 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T17:56:21.237 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-16T17:56:21.237 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T17:56:21.394 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-16T17:56:21.433 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-16T17:56:21.466 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-16T17:56:21.525 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-16T17:56:21.568 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-16T17:56:21.696 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T17:56:21.951 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-16T17:56:21.951 INFO:teuthology.orchestra.run.smithi090.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-16T17:56:21.951 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:28.617 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 1/198 2024-03-16T17:56:28.617 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2/198 2024-03-16T17:56:28.617 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 3/198 2024-03-16T17:56:28.617 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 4/198 2024-03-16T17:56:28.617 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1681.gaba3c 5/198 2024-03-16T17:56:28.617 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 6/198 2024-03-16T17:56:28.617 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 7/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 8/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 9/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 10/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 11/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 12/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 13/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 14/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_ 15/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 16/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 17/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_6 18/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86 19/198 2024-03-16T17:56:28.618 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 20/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el 21/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9. 22/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_6 23/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 24/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 25/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 26/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 27/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 28/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 29/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2. 30/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noa 31/198 2024-03-16T17:56:28.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.n 32/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba 33/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el 34/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 35/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.e 36/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 37/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 38/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 39/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-16T17:56:28.620 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-16T17:56:28.621 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-16T17:56:28.622 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-16T17:56:28.623 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-16T17:56:28.624 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-16T17:56:28.625 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-16T17:56:28.626 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-16T17:56:28.627 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 151/198 2024-03-16T17:56:28.628 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 153/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 157/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 159/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 161/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 163/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-16T17:56:28.629 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-16T17:56:28.630 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-16T17:56:28.631 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-16T17:56:28.632 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-16T17:56:28.632 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-16T17:56:28.632 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-16T17:56:28.632 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-16T17:56:31.158 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-16T17:56:31.158 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:31.158 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-03-16T17:56:31.158 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: acl-2.3.1-4.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.159 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: automake-1.16.2-8.el9.noarch 2024-03-16T17:56:31.160 INFO:teuthology.orchestra.run.smithi090.stdout: bison-3.7.4-5.el9.x86_64 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:56:31.161 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top-2:19.0.0-1681.gaba3c1d2.el9.noarch 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: flex-2.6.4-9.el9.x86_64 2024-03-16T17:56:31.162 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: jq-1.6-16.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.163 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-16T17:56:31.164 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-13.11-1.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-16T17:56:31.165 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-16T17:56:31.166 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-16T17:56:31.167 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-16T17:56:31.168 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-16T17:56:31.169 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-16T17:56:31.170 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.171 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.171 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.0.0-1681.gaba3c1d2.el9.x86_64 2024-03-16T17:56:31.171 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-03-16T17:56:31.171 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-16T17:56:31.171 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-16T17:56:31.171 INFO:teuthology.orchestra.run.smithi090.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-16T17:56:31.171 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-16T17:56:31.171 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-16T17:56:31.171 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-16T17:56:31.171 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:31.172 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-03-16T17:56:32.008 DEBUG:teuthology.parallel:result is None 2024-03-16T17:56:32.008 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-16T17:56:32.259 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T17:56:32.324 INFO:teuthology.orchestra.run.smithi082.stdout:19.0.0-1681.gaba3c1d2.el9 2024-03-16T17:56:32.325 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1681.gaba3c1d2.el9 2024-03-16T17:56:32.325 INFO:teuthology.task.install:The correct ceph version 19.0.0-1681.gaba3c1d2 is installed. 2024-03-16T17:56:32.327 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-16T17:56:32.465 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-16T17:56:32.521 INFO:teuthology.orchestra.run.smithi090.stdout:19.0.0-1681.gaba3c1d2.el9 2024-03-16T17:56:32.521 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1681.gaba3c1d2.el9 2024-03-16T17:56:32.521 INFO:teuthology.task.install:The correct ceph version 19.0.0-1681.gaba3c1d2 is installed. 2024-03-16T17:56:32.523 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-16T17:56:32.523 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:32.523 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T17:56:32.624 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:56:32.624 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T17:56:32.701 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-16T17:56:32.701 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:32.701 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T17:56:32.801 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T17:56:32.935 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:56:32.935 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T17:56:33.008 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T17:56:33.137 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-16T17:56:33.138 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:33.138 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T17:56:33.242 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T17:56:33.377 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:56:33.377 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T17:56:33.447 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T17:56:33.555 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-16T17:56:33.556 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:33.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T17:56:33.654 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T17:56:33.748 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:56:33.748 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T17:56:33.816 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T17:56:33.924 INFO:teuthology.run_tasks:Running task ceph... 2024-03-16T17:56:34.008 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-16T17:56:34.008 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /var/log/ceph 2024-03-16T17:56:34.016 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /var/log/ceph 2024-03-16T17:56:34.102 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-16T17:56:34.102 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-16T17:56:34.107 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-16T17:56:34.187 INFO:tasks.ceph:Creating extra log directories... 2024-03-16T17:56:34.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-16T17:56:34.194 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-16T17:56:34.288 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-16T17:56:34.288 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-16T17:56:34.288 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-16T17:56:34.289 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:34.289 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-16T17:56:34.341 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-16T17:56:34.607 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-16T17:56:34.702 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:56:34.703 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-16T17:56:34.741 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-16T17:56:34.982 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-16T17:56:35.080 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-16T17:56:35.080 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-16T17:56:35.080 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024', 'branch': 'wip-lusov-qdb-squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/acls}', 'email': None, 'first_in_suite': False, 'job_id': '7602024', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-qdb-squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-qdb-squid', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6918, 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'sleep_before_teardown': 0, 'subset': '111/10000', 'suite': 'fs', 'suite_branch': 'wip-lusov-qdb-squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi082.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMWDTjOWyp/xznfLFtNdGdjRytqkO/3ebZ6djnkvUAL2aYk9ZxFLqqX4KHBa7qOMy3OriVHGpGWzJ619MA+ZWK4=', 'smithi090.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMHGJiu0M9WpvHJtGRtV6+Bjf+tQAbbC+q4LpxhEK8u3bCpPcuXOnJC7Kux/5hu6k9cb6uWa/Jm96mHFtJ48DbI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_acls']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-15_11:44:52', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-16T17:56:35.081 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-16T17:56:35.088 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-16T17:56:35.139 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/run/ceph 2024-03-16T17:56:35.144 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m0777 -- /var/run/ceph 2024-03-16T17:56:35.235 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:35.235 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T17:56:35.289 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T17:56:35.290 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-03-16T17:56:35.339 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T17:56:35.339 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T17:56:35.339 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-16T17:56:35.339 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T17:56:35.339 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T17:56:35.339 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-16 17:55:44.988012332 +0000 2024-03-16T17:56:35.339 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-16 17:53:18.547543936 +0000 2024-03-16T17:56:35.339 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-16 17:53:18.547543936 +0000 2024-03-16T17:56:35.340 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-03-16 17:53:18.547543936 +0000 2024-03-16T17:56:35.340 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T17:56:35.426 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-16T17:56:35.427 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-16T17:56:35.427 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000331949 s, 1.5 MB/s 2024-03-16T17:56:35.431 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T17:56:35.487 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-03-16T17:56:35.536 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T17:56:35.537 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T17:56:35.537 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-16T17:56:35.537 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T17:56:35.537 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T17:56:35.537 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-16 17:55:44.989012314 +0000 2024-03-16T17:56:35.537 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-16 17:53:19.107533594 +0000 2024-03-16T17:56:35.537 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-16 17:53:19.107533594 +0000 2024-03-16T17:56:35.537 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-03-16 17:53:19.107533594 +0000 2024-03-16T17:56:35.537 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T17:56:35.615 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-16T17:56:35.615 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-16T17:56:35.615 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000310063 s, 1.7 MB/s 2024-03-16T17:56:35.618 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T17:56:35.672 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-03-16T17:56:35.719 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T17:56:35.719 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T17:56:35.719 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-16T17:56:35.719 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T17:56:35.720 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T17:56:35.720 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-16 17:55:44.990012296 +0000 2024-03-16T17:56:35.720 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-16 17:53:19.671523179 +0000 2024-03-16T17:56:35.720 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-16 17:53:19.671523179 +0000 2024-03-16T17:56:35.720 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-03-16 17:53:19.671523179 +0000 2024-03-16T17:56:35.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T17:56:35.845 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-16T17:56:35.846 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-16T17:56:35.846 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.0003228 s, 1.6 MB/s 2024-03-16T17:56:35.850 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T17:56:35.904 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-03-16T17:56:35.953 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T17:56:35.953 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T17:56:35.953 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-03-16T17:56:35.953 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T17:56:35.953 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T17:56:35.953 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-16 17:55:44.990012296 +0000 2024-03-16T17:56:35.954 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-16 17:53:20.212513188 +0000 2024-03-16T17:56:35.954 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-16 17:53:20.212513188 +0000 2024-03-16T17:56:35.954 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-03-16 17:53:20.212513188 +0000 2024-03-16T17:56:35.954 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T17:56:36.041 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-03-16T17:56:36.041 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-03-16T17:56:36.041 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000341298 s, 1.5 MB/s 2024-03-16T17:56:36.045 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T17:56:36.101 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T17:56:36.101 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:56:36.101 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T17:56:36.154 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T17:56:36.155 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-03-16T17:56:36.202 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T17:56:36.203 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T17:56:36.203 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-16T17:56:36.203 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T17:56:36.203 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T17:56:36.203 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-16 17:56:22.372925346 +0000 2024-03-16T17:56:36.203 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-16 17:53:22.357020519 +0000 2024-03-16T17:56:36.203 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-16 17:53:22.357020519 +0000 2024-03-16T17:56:36.203 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-03-16 17:53:22.357020519 +0000 2024-03-16T17:56:36.203 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T17:56:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-16T17:56:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-16T17:56:36.291 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000325018 s, 1.6 MB/s 2024-03-16T17:56:36.295 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T17:56:36.350 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-03-16T17:56:36.398 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T17:56:36.398 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T17:56:36.398 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-03-16T17:56:36.398 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T17:56:36.398 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T17:56:36.398 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-16 17:56:22.373925330 +0000 2024-03-16T17:56:36.399 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-16 17:53:22.913011723 +0000 2024-03-16T17:56:36.399 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-16 17:53:22.913011723 +0000 2024-03-16T17:56:36.399 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-03-16 17:53:22.913011723 +0000 2024-03-16T17:56:36.399 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T17:56:36.483 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-16T17:56:36.483 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-16T17:56:36.483 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000342841 s, 1.5 MB/s 2024-03-16T17:56:36.488 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T17:56:36.542 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-03-16T17:56:36.592 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T17:56:36.592 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T17:56:36.592 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-03-16T17:56:36.592 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T17:56:36.592 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T17:56:36.592 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-16 17:56:22.374925314 +0000 2024-03-16T17:56:36.593 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-16 17:53:23.469002927 +0000 2024-03-16T17:56:36.593 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-16 17:53:23.469002927 +0000 2024-03-16T17:56:36.593 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-03-16 17:53:23.469002927 +0000 2024-03-16T17:56:36.593 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T17:56:36.671 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-16T17:56:36.672 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-16T17:56:36.672 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000293092 s, 1.7 MB/s 2024-03-16T17:56:36.674 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T17:56:36.724 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-03-16T17:56:36.771 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T17:56:36.771 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T17:56:36.771 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-16T17:56:36.771 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T17:56:36.771 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-03-16T17:56:36.771 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-03-16 17:56:22.374925314 +0000 2024-03-16T17:56:36.771 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-03-16 17:53:24.030994036 +0000 2024-03-16T17:56:36.772 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-03-16 17:53:24.030994036 +0000 2024-03-16T17:56:36.772 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-03-16 17:53:24.030994036 +0000 2024-03-16T17:56:36.772 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T17:56:36.907 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-03-16T17:56:36.907 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-03-16T17:56:36.907 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000373723 s, 1.4 MB/s 2024-03-16T17:56:36.911 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T17:56:36.970 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T17:56:36.970 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi090.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-16T17:56:36.970 INFO:tasks.ceph:Generating config... 2024-03-16T17:56:36.971 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-16T17:56:36.971 INFO:tasks.ceph:[client] debug client = 20 2024-03-16T17:56:36.971 INFO:tasks.ceph:[client] debug ms = 1 2024-03-16T17:56:36.972 INFO:tasks.ceph:[client] pid file = 2024-03-16T17:56:36.972 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-16T17:56:36.972 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-16T17:56:36.972 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-16T17:56:36.972 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-16T17:56:36.973 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-16T17:56:36.973 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-16T17:56:36.973 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-16T17:56:36.973 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-16T17:56:36.973 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-16T17:56:36.973 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-16T17:56:36.973 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-16T17:56:36.973 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-16T17:56:36.973 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-16T17:56:36.973 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-16T17:56:36.973 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-16T17:56:36.973 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-16T17:56:36.973 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-16T17:56:36.973 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-16T17:56:36.973 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-16T17:56:36.974 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-16T17:56:36.974 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-16T17:56:36.974 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-16T17:56:36.974 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-16T17:56:36.974 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-16T17:56:36.974 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-16T17:56:36.974 INFO:tasks.ceph:Setting up mon.a... 2024-03-16T17:56:36.974 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-16T17:56:37.115 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.keyring 2024-03-16T17:56:37.124 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-16T17:56:37.249 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-16T17:56:37.332 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.82'), ('mon.b', '172.21.15.90'), ('mon.c', '[v2:172.21.15.90:3301,v1:172.21.15.90:6790]')] 2024-03-16T17:56:37.333 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.82,172.21.15.90,[v2:172.21.15.90:3301,v1:172.21.15.90:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-16T17:56:37.333 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:37.333 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-16T17:56:37.509 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.82 --add b 172.21.15.90 --addv c '[v2:172.21.15.90:3301,v1:172.21.15.90:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:56:37.622 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:56:37.622 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: generated fsid 08082c2e-06a4-4a16-ab2e-9e9e244a6661 2024-03-16T17:56:37.622 INFO:teuthology.orchestra.run.smithi082.stdout:setting min_mon_release = quincy 2024-03-16T17:56:37.622 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 0 2024-03-16T17:56:37.623 INFO:teuthology.orchestra.run.smithi082.stdout:fsid 08082c2e-06a4-4a16-ab2e-9e9e244a6661 2024-03-16T17:56:37.623 INFO:teuthology.orchestra.run.smithi082.stdout:last_changed 2024-03-16T17:56:37.619841+0000 2024-03-16T17:56:37.623 INFO:teuthology.orchestra.run.smithi082.stdout:created 2024-03-16T17:56:37.619841+0000 2024-03-16T17:56:37.623 INFO:teuthology.orchestra.run.smithi082.stdout:min_mon_release 17 (quincy) 2024-03-16T17:56:37.623 INFO:teuthology.orchestra.run.smithi082.stdout:election_strategy: 1 2024-03-16T17:56:37.623 INFO:teuthology.orchestra.run.smithi082.stdout:0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-03-16T17:56:37.623 INFO:teuthology.orchestra.run.smithi082.stdout:1: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.b 2024-03-16T17:56:37.623 INFO:teuthology.orchestra.run.smithi082.stdout:2: [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] mon.c 2024-03-16T17:56:37.623 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-16T17:56:37.626 DEBUG:teuthology.orchestra.run.smithi082:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-16T17:56:37.677 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 08082c2e-06a4-4a16-ab2e-9e9e244a6661... 2024-03-16T17:56:37.677 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-16T17:56:37.682 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-16T17:56:37.836 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-03-16T17:56:37.836 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.837 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-03-16T17:56:37.838 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-03-16T17:56:37.839 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.82,172.21.15.90,[v2:172.21.15.90:3301,v1:172.21.15.90:6790]" 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on skipped message = False 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 08082c2e-06a4-4a16-ab2e-9e9e244a6661 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-03-16T17:56:37.840 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-03-16T17:56:37.841 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: bdev async discard = True 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: bdev enable discard = True 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore allocator = bitmap 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 1/20 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 1/20 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 4/10 2024-03-16T17:56:37.842 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-03-16T17:56:37.843 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-16T17:56:37.844 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-03-16T17:56:37.845 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-03-16T17:56:37.846 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-03-16T17:56:37.847 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-03-16T17:56:37.847 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-03-16T17:56:37.847 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-03-16T17:56:37.847 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-03-16T17:56:37.847 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-03-16T17:56:37.847 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-03-16T17:56:37.847 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-03-16T17:56:37.847 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-03-16T17:56:37.848 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = True 2024-03-16T17:56:37.848 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-03-16T17:56:37.848 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = True 2024-03-16T17:56:37.848 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-03-16T17:56:37.848 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-03-16T17:56:37.848 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-03-16T17:56:37.856 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-03-16T17:56:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: chdir = "" 2024-03-16T17:56:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-16T17:56:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: auth supported = cephx 2024-03-16T17:56:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: filestore xattr use omap = true 2024-03-16T17:56:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: mon clock drift allowed = 1.000 2024-03-16T17:56:37.856 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: osd crush chooseleaf type = 0 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: auth debug = true 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on old message = true 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on bug = true 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: mon pg warn max object skew = 0 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: # disable pg_autoscaler by default for new pools 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.857 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default size = 2 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow primary affinity = true 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd allow pg remap = true 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on legacy crush tunables = false 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on crush straw calc version zero = false 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on no sortbitwise = false 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on osd down out interval zero = false 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: mon warn on too few osds = false 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: mon_allow_pool_size_one = true 2024-03-16T17:56:37.858 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: osd default data pool replay window = 5 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: mon allow pool delete = true 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: mon cluster log file level = debug 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: debug asserts on shutdown = true 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: mon health detail to clog = false 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: mon host = "172.21.15.82,172.21.15.90,[v2:172.21.15.90:3301,v1:172.21.15.90:6790]" 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: ms die on skipped message = False 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = 08082c2e-06a4-4a16-ab2e-9e9e244a6661 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.859 INFO:teuthology.orchestra.run.smithi090.stdout:[osd] 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd journal size = 100 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub load threshold = 5.0 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd scrub max interval = 600 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd mclock profile = high_recovery_ops 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd recover clone overlap = true 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd recovery max chunk = 1048576 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug shutdown = true 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug op order = true 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug verify stray on activate = true 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug trim objects = true 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.860 INFO:teuthology.orchestra.run.smithi090.stdout: osd open classes on start = true 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug pg log writeout = true 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: osd deep scrub update digest min age = 30 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: osd map max advance = 10 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: journal zero on create = true 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: filestore ondisk finisher threads = 3 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: filestore apply finisher threads = 3 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: bdev debug aio = true 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: osd debug misdirected ops = true 2024-03-16T17:56:37.861 INFO:teuthology.orchestra.run.smithi090.stdout: bdev async discard = True 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: bdev enable discard = True 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore allocator = bitmap 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore block size = 96636764160 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: bluestore fsck on mount = True 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluefs = 1/20 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: debug bluestore = 1/20 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: debug osd = 20 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: debug rocksdb = 4/10 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd backfillfull_ratio = 0.85 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd full ratio = 0.9 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd nearfull ratio = 0.8 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: osd failsafe full ratio = 0.95 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: osd objectstore = bluestore 2024-03-16T17:56:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: osd op complaint time = 180 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr] 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: debug mgr = 20 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: mgr/telemetry/nag = false 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout:[mon] 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: debug mon = 20 2024-03-16T17:56:37.863 INFO:teuthology.orchestra.run.smithi090.stdout: debug paxos = 20 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: debug auth = 20 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: mon data avail warn = 5 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: mon mgr mkfs grace = 240 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min pgs per osd = 4 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd reporter subtree level = osd 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: mon osd prime pg temp = true 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: mon reweight min bytes per osd = 10 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: auth mon ticket ttl = 660 # 11m 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: auth service ticket ttl = 240 # 4m 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: # don't complain about insecure global_id in the test suite 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-16T17:56:37.864 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: # 1m isn't quite enough 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: mon_down_mkfs_grace = 2m 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: mon_warn_on_filestore_osds = false 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: mon op complaint time = 120 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout:[client] 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: rgw cache enabled = true 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable ops log = true 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: rgw enable usage log = true 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: client mount timeout = 600 2024-03-16T17:56:37.865 INFO:teuthology.orchestra.run.smithi090.stdout: debug client = 20 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: pid file = "" 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: rados mon op timeout = 900 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout: rados osd op timeout = 900 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.a] 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.b] 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout:[mon.c] 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a] 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c] 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b] 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d] 2024-03-16T17:56:37.866 INFO:teuthology.orchestra.run.smithi090.stdout:[mds] 2024-03-16T17:56:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: debug mds = 20 2024-03-16T17:56:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: debug mds balancer = 20 2024-03-16T17:56:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: debug ms = 1 2024-03-16T17:56:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: mds debug frag = True 2024-03-16T17:56:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: mds debug scatterstat = True 2024-03-16T17:56:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: mds op complaint time = 180 2024-03-16T17:56:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: mds verify scatter = True 2024-03-16T17:56:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: osd op complaint time = 180 2024-03-16T17:56:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: rados mon op timeout = 900 2024-03-16T17:56:37.867 INFO:teuthology.orchestra.run.smithi090.stdout: rados osd op timeout = 900 2024-03-16T17:56:37.900 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-16T17:56:37.901 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-16T17:56:38.045 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-16T17:56:38.045 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:38.045 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-16T17:56:38.094 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:38.094 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-16T17:56:38.190 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T17:56:38.190 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:38.190 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-16T17:56:38.190 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-16T17:56:38.327 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:56:38.327 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:56:38.424 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T17:56:38.424 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:56:38.424 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-16T17:56:38.425 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-16T17:56:38.577 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:56:38.577 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:56:38.675 INFO:tasks.ceph:Setting up mon nodes... 2024-03-16T17:56:38.676 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-16T17:56:38.676 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-16T17:56:38.858 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-16T17:56:38.868 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-16T17:56:39.060 INFO:teuthology.orchestra.run.smithi090.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-16T17:56:39.071 INFO:tasks.ceph:Setting up mds nodes... 2024-03-16T17:56:39.071 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-16T17:56:39.255 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-16T17:56:39.264 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-16T17:56:39.359 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-16T17:56:39.535 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-16T17:56:39.544 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-16T17:56:39.628 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-16T17:56:39.809 INFO:teuthology.orchestra.run.smithi090.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-16T17:56:39.819 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-16T17:56:39.915 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-16T17:56:40.087 INFO:teuthology.orchestra.run.smithi090.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-16T17:56:40.096 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-16T17:56:40.192 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-16T17:56:40.192 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-16T17:56:40.320 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-16T17:56:40.376 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-16T17:56:40.501 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-16T17:56:40.558 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-16T17:56:40.683 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-16T17:56:40.732 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-16T17:56:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-16T17:56:40.911 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-16T17:56:40.911 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi090.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-16T17:56:40.912 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-16T17:56:41.005 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T17:56:41.005 INFO:tasks.ceph:role: osd.0 2024-03-16T17:56:41.006 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T17:56:41.006 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-16T17:56:41.105 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T17:56:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T17:56:41.105 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T17:56:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-16T17:56:41.106 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T17:56:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-03-16T17:56:41.106 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T17:56:41.106 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-16T17:56:41.106 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T17:56:41.106 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T17:56:41.625 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-03-16T17:56:41.631 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-03-16T17:56:41.631 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-16T17:56:41.819 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-16T17:56:41.918 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-16T17:56:42.001 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T17:56:42.001 INFO:tasks.ceph:role: osd.1 2024-03-16T17:56:42.001 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T17:56:42.001 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-16T17:56:42.088 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T17:56:42.088 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T17:56:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T17:56:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-16T17:56:42.089 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T17:56:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-03-16T17:56:42.089 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T17:56:42.089 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-16T17:56:42.089 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T17:56:42.090 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T17:56:42.565 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-03-16T17:56:42.570 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-03-16T17:56:42.571 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-16T17:56:42.689 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-16T17:56:42.777 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-16T17:56:42.855 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T17:56:42.855 INFO:tasks.ceph:role: osd.2 2024-03-16T17:56:42.855 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T17:56:42.855 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-16T17:56:42.941 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T17:56:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T17:56:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T17:56:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-16T17:56:42.941 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T17:56:42.941 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-03-16T17:56:42.941 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T17:56:42.942 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-16T17:56:42.942 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T17:56:42.942 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T17:56:43.466 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-03-16T17:56:43.473 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-03-16T17:56:43.473 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-16T17:56:43.661 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-16T17:56:43.763 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-16T17:56:43.857 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T17:56:43.857 INFO:tasks.ceph:role: osd.3 2024-03-16T17:56:43.857 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T17:56:43.857 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-16T17:56:43.952 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T17:56:43.952 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T17:56:43.952 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T17:56:43.952 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-16T17:56:43.952 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T17:56:43.953 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-03-16T17:56:43.953 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T17:56:43.953 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-16T17:56:43.953 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T17:56:43.953 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T17:56:44.418 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-03-16T17:56:44.425 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-03-16T17:56:44.425 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-16T17:56:44.569 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-16T17:56:44.658 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:56:44.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:44.805+0000 7efc12f49640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-16T17:56:44.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:44.806+0000 7efc12f49640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-16T17:56:44.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:44.806+0000 7efc12f49640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-16T17:56:44.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:44.806+0000 7efc12f49640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-16T17:56:44.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:44.806+0000 7efc12f49640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-16T17:56:44.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:44.811+0000 7efc12f49640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-16T17:56:47.996 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T17:56:48.088 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:56:48.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:48.221+0000 7fab4a61b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-16T17:56:48.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:48.222+0000 7fab4a61b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-16T17:56:48.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:48.223+0000 7fab4a61b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-16T17:56:48.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:48.223+0000 7fab4a61b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-16T17:56:48.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:48.223+0000 7fab4a61b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-16T17:56:48.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:48.225+0000 7fab4a61b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-16T17:56:51.432 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T17:56:51.529 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:56:51.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:51.666+0000 7fbdd0bbb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-16T17:56:51.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:51.667+0000 7fbdd0bbb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-16T17:56:51.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:51.667+0000 7fbdd0bbb640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-16T17:56:51.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:51.667+0000 7fbdd0bbb640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-16T17:56:51.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:51.667+0000 7fbdd0bbb640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-16T17:56:51.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:51.669+0000 7fbdd0bbb640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-16T17:56:54.850 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T17:56:54.944 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:56:55.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:55.072+0000 7f7edc815640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-16T17:56:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:55.073+0000 7f7edc815640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-16T17:56:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:55.073+0000 7f7edc815640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-16T17:56:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:55.073+0000 7f7edc815640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-16T17:56:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:55.073+0000 7f7edc815640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-16T17:56:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:56:55.075+0000 7f7edc815640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-16T17:56:58.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T17:56:58.581 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-16T17:56:58.681 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T17:56:58.682 INFO:tasks.ceph:role: osd.4 2024-03-16T17:56:58.682 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T17:56:58.682 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-16T17:56:58.779 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T17:56:58.779 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T17:56:58.779 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T17:56:58.779 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-16T17:56:58.779 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T17:56:58.780 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2024-03-16T17:56:58.780 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T17:56:58.780 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-16T17:56:58.780 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T17:56:58.780 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T17:56:59.306 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2024-03-16T17:56:59.313 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2024-03-16T17:56:59.313 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-16T17:56:59.496 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-16T17:56:59.605 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-16T17:56:59.689 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T17:56:59.689 INFO:tasks.ceph:role: osd.5 2024-03-16T17:56:59.689 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T17:56:59.690 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-16T17:56:59.774 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T17:56:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T17:56:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T17:56:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-16T17:56:59.775 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T17:56:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2024-03-16T17:56:59.775 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T17:56:59.775 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-16T17:56:59.775 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T17:56:59.775 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T17:57:00.247 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2024-03-16T17:57:00.253 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2024-03-16T17:57:00.253 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-16T17:57:00.380 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-16T17:57:00.468 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-16T17:57:00.550 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T17:57:00.551 INFO:tasks.ceph:role: osd.6 2024-03-16T17:57:00.551 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T17:57:00.551 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-16T17:57:00.625 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T17:57:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T17:57:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T17:57:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-16T17:57:00.625 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T17:57:00.625 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2024-03-16T17:57:00.625 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T17:57:00.625 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-16T17:57:00.626 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T17:57:00.626 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T17:57:01.160 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2024-03-16T17:57:01.165 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2024-03-16T17:57:01.165 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-16T17:57:01.363 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-16T17:57:01.455 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-16T17:57:01.546 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T17:57:01.546 INFO:tasks.ceph:role: osd.7 2024-03-16T17:57:01.546 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T17:57:01.547 DEBUG:teuthology.orchestra.run.smithi090:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-16T17:57:01.645 INFO:teuthology.orchestra.run.smithi090.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T17:57:01.645 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T17:57:01.645 INFO:teuthology.orchestra.run.smithi090.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T17:57:01.645 INFO:teuthology.orchestra.run.smithi090.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-16T17:57:01.645 INFO:teuthology.orchestra.run.smithi090.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T17:57:01.645 INFO:teuthology.orchestra.run.smithi090.stdout: = sunit=0 swidth=0 blks 2024-03-16T17:57:01.645 INFO:teuthology.orchestra.run.smithi090.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T17:57:01.645 INFO:teuthology.orchestra.run.smithi090.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-16T17:57:01.646 INFO:teuthology.orchestra.run.smithi090.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T17:57:01.646 INFO:teuthology.orchestra.run.smithi090.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T17:57:02.097 INFO:teuthology.orchestra.run.smithi090.stdout:Discarding blocks...Done. 2024-03-16T17:57:02.103 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi090.front.sepia.ceph.com -o noatime 2024-03-16T17:57:02.103 DEBUG:teuthology.orchestra.run.smithi090:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-16T17:57:02.247 DEBUG:teuthology.orchestra.run.smithi090:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-16T17:57:02.336 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:57:02.472 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:02.468+0000 7fadede48640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-16T17:57:02.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:02.471+0000 7fadede48640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-16T17:57:02.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:02.471+0000 7fadede48640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-16T17:57:02.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:02.471+0000 7fadede48640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-16T17:57:02.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:02.471+0000 7fadede48640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-16T17:57:02.481 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:02.477+0000 7fadede48640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-16T17:57:05.082 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:57:05.090 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:57:05.653 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T17:57:05.748 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:57:05.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:05.881+0000 7f120d385640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-16T17:57:05.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:05.881+0000 7f120d385640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-16T17:57:05.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:05.881+0000 7f120d385640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-16T17:57:05.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:05.881+0000 7f120d385640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-16T17:57:05.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:05.881+0000 7f120d385640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-16T17:57:05.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:05.883+0000 7f120d385640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-16T17:57:08.816 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T17:57:08.914 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:57:09.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:09.048+0000 7fd586e20640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-16T17:57:09.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:09.048+0000 7fd586e20640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-16T17:57:09.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:09.048+0000 7fd586e20640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-16T17:57:09.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:09.048+0000 7fd586e20640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-16T17:57:09.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:09.048+0000 7fd586e20640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-16T17:57:09.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:09.051+0000 7fd586e20640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-16T17:57:11.980 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-16T17:57:12.071 DEBUG:teuthology.orchestra.run.smithi090:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:57:12.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:12.205+0000 7f8f47ab1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-16T17:57:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:12.205+0000 7f8f47ab1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-16T17:57:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:12.206+0000 7f8f47ab1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-16T17:57:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:12.206+0000 7f8f47ab1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-16T17:57:12.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:12.206+0000 7f8f47ab1640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-16T17:57:12.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:12.208+0000 7f8f47ab1640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-16T17:57:15.150 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-16T17:57:15.245 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-16T17:57:15.246 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:15.246 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-16T17:57:15.346 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:15.346 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-16T17:57:15.440 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:15.440 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-16T17:57:15.537 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:15.537 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-16T17:57:15.631 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:15.631 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-16T17:57:15.720 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:15.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-16T17:57:15.857 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:15.857 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-16T17:57:15.951 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:15.951 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-16T17:57:16.045 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:16.045 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-16T17:57:16.141 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:16.141 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-16T17:57:16.232 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:16.232 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-16T17:57:16.328 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:16.328 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-16T17:57:16.412 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:16.412 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-16T17:57:16.548 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:16.548 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-16T17:57:16.638 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:16.638 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-16T17:57:16.694 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:16.695 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-16T17:57:16.749 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:16.749 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-16T17:57:16.805 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:16.805 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-16T17:57:16.857 INFO:tasks.ceph:Adding keys to all mons... 2024-03-16T17:57:16.857 DEBUG:teuthology.orchestra.run.smithi082:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-16T17:57:16.864 DEBUG:teuthology.orchestra.run.smithi090:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-16T17:57:16.945 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.y] 2024-03-16T17:57:16.945 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBW3fVlIaX+MhAAN42Ezb1GwDDzSV3Nx1ZcbQ== 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBX3fVlBK4TDxAAgYBMjjtLq7Ts2ELUHOfrTw== 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBX3fVlZqfLHxAAr8uX4ldhnbNd3HwZwM3Ljw== 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.0] 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBc3fVlYSYRMBAA1Tc0pI8roY4iOYXJLRb3yQ== 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.1] 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBg3fVlQYBLDRAAx8fhhK/pE3YSmSzTk0TC6A== 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.2] 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBj3fVlRNzLJxAAF2Kr2rtYnU1s3NRxt/w6dg== 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.3] 2024-03-16T17:57:16.946 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBn3fVln2BkBBAAkAPCd6Z4ShuJUtZx62MmHA== 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.x] 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBX3fVl1ZlxAxAAHllQAsGYtDLSy1iO0jKGug== 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBX3fVlKwweMBAAlSH59V+WRQ7pPFGRQihe0g== 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBY3fVl1XMTBRAAnwrP8FPBLSLM+/rR39Vu7Q== 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.4] 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBu3fVlGZ38GxAAtcYQv0Efbj5ay4vsakaAjQ== 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.5] 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBx3fVlyRiYNBAArUz3DTmHVPqCm+BXw0W5vg== 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.6] 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQB13fVl7NTiAhAAJFQwOdPJqIKyeRencS6JPw== 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.7] 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQB43fVl87VCDBAAg8jmXVeiQskKjhT2pSHNuA== 2024-03-16T17:57:16.947 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-03-16T17:57:16.948 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBY3fVlVabyEhAAXRXAiI2UCfBSDg0N+ezAyw== 2024-03-16T17:57:16.948 INFO:teuthology.orchestra.run.smithi082.stdout:[client.2] 2024-03-16T17:57:16.948 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBY3fVlkjG3HRAAn0f/SF1SsTzAJ0wRpePYsA== 2024-03-16T17:57:16.948 INFO:teuthology.orchestra.run.smithi082.stdout:[client.1] 2024-03-16T17:57:16.948 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBY3fVlV/uMKBAAbU4pVWf+Fs33591f43297Q== 2024-03-16T17:57:16.948 INFO:teuthology.orchestra.run.smithi082.stdout:[client.3] 2024-03-16T17:57:16.948 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBY3fVlMOHVMhAAuOUJ6OGv536nnAmMLACc8w== 2024-03-16T17:57:16.948 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr.y] 2024-03-16T17:57:16.948 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBW3fVlIaX+MhAAN42Ezb1GwDDzSV3Nx1ZcbQ== 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a] 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBX3fVlBK4TDxAAgYBMjjtLq7Ts2ELUHOfrTw== 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c] 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBX3fVlZqfLHxAAr8uX4ldhnbNd3HwZwM3Ljw== 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.0] 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBc3fVlYSYRMBAA1Tc0pI8roY4iOYXJLRb3yQ== 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.1] 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBg3fVlQYBLDRAAx8fhhK/pE3YSmSzTk0TC6A== 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.2] 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBj3fVlRNzLJxAAF2Kr2rtYnU1s3NRxt/w6dg== 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.3] 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBn3fVln2BkBBAAkAPCd6Z4ShuJUtZx62MmHA== 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout:[mgr.x] 2024-03-16T17:57:16.949 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBX3fVl1ZlxAxAAHllQAsGYtDLSy1iO0jKGug== 2024-03-16T17:57:16.950 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b] 2024-03-16T17:57:16.950 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBX3fVlKwweMBAAlSH59V+WRQ7pPFGRQihe0g== 2024-03-16T17:57:16.950 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d] 2024-03-16T17:57:16.950 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBY3fVl1XMTBRAAnwrP8FPBLSLM+/rR39Vu7Q== 2024-03-16T17:57:16.950 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.4] 2024-03-16T17:57:16.950 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBu3fVlGZ38GxAAtcYQv0Efbj5ay4vsakaAjQ== 2024-03-16T17:57:16.950 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.5] 2024-03-16T17:57:16.950 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBx3fVlyRiYNBAArUz3DTmHVPqCm+BXw0W5vg== 2024-03-16T17:57:16.950 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.6] 2024-03-16T17:57:16.950 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQB13fVl7NTiAhAAJFQwOdPJqIKyeRencS6JPw== 2024-03-16T17:57:16.951 INFO:teuthology.orchestra.run.smithi090.stdout:[osd.7] 2024-03-16T17:57:16.951 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQB43fVl87VCDBAAg8jmXVeiQskKjhT2pSHNuA== 2024-03-16T17:57:16.951 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-03-16T17:57:16.951 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBY3fVlVabyEhAAXRXAiI2UCfBSDg0N+ezAyw== 2024-03-16T17:57:16.951 INFO:teuthology.orchestra.run.smithi090.stdout:[client.2] 2024-03-16T17:57:16.951 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBY3fVlkjG3HRAAn0f/SF1SsTzAJ0wRpePYsA== 2024-03-16T17:57:16.951 INFO:teuthology.orchestra.run.smithi090.stdout:[client.1] 2024-03-16T17:57:16.951 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBY3fVlV/uMKBAAbU4pVWf+Fs33591f43297Q== 2024-03-16T17:57:16.951 INFO:teuthology.orchestra.run.smithi090.stdout:[client.3] 2024-03-16T17:57:16.951 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQBY3fVlMOHVMhAAuOUJ6OGv536nnAmMLACc8w== 2024-03-16T17:57:16.953 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T17:57:17.000 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T17:57:17.144 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T17:57:17.150 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T17:57:17.290 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T17:57:17.297 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T17:57:17.437 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:17.441 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:17.582 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:17.587 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:17.730 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:17.736 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:17.876 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:17.880 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:18.020 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T17:57:18.031 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T17:57:18.171 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T17:57:18.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T17:57:18.322 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T17:57:18.329 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T17:57:18.471 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:18.478 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:18.621 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:18.626 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:18.763 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:18.770 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:18.910 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:18.917 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T17:57:19.059 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T17:57:19.067 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T17:57:19.204 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T17:57:19.211 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T17:57:19.361 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T17:57:19.367 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T17:57:19.508 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T17:57:19.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T17:57:19.657 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-16T17:57:19.657 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-16T17:57:19.747 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-16T17:57:19.899 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-16T17:57:19.983 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-16T17:57:20.078 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-16T17:57:20.237 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-16T17:57:20.316 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-16T17:57:20.406 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-16T17:57:20.556 DEBUG:teuthology.orchestra.run.smithi090:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-16T17:57:20.641 DEBUG:teuthology.orchestra.run.smithi082:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:57:20.649 DEBUG:teuthology.orchestra.run.smithi090:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-16T17:57:20.700 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-16T17:57:20.700 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-16T17:57:20.701 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-16T17:57:20.708 INFO:tasks.ceph.mon.a:Started 2024-03-16T17:57:20.708 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-16T17:57:20.708 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-16T17:57:20.714 INFO:tasks.ceph.mon.b:Started 2024-03-16T17:57:20.714 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-16T17:57:20.714 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-16T17:57:20.721 INFO:tasks.ceph.mon.c:Started 2024-03-16T17:57:20.721 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-16T17:57:20.721 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-16T17:57:20.721 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-16T17:57:20.728 INFO:tasks.ceph.mgr.y:Started 2024-03-16T17:57:20.728 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-16T17:57:20.728 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-16T17:57:20.733 INFO:tasks.ceph.mgr.x:Started 2024-03-16T17:57:20.733 INFO:tasks.ceph:Setting crush tunables to default 2024-03-16T17:57:20.734 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd crush tunables default 2024-03-16T17:57:20.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.904+0000 7f6967c00640 1 Processor -- start 2024-03-16T17:57:20.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.904+0000 7f6967c00640 1 -- start start 2024-03-16T17:57:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.905+0000 7f6967c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 0x7f696015e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.905+0000 7f6967c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69600ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6967c00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69600b33c0 con 0x7f69600d48e0 2024-03-16T17:57:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6967c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69600d1cd0 con 0x7f696015e040 2024-03-16T17:57:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6967c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69600d1fb0 con 0x7f69600d2ab0 2024-03-16T17:57:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 0x7f696015e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 0x7f696015e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38570/0 (socket says 172.21.15.82:38570) 2024-03-16T17:57:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965200640 1 -- 172.21.15.82:0/630638991 learned_addr learned my addr 172.21.15.82:0/630638991 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965200640 1 -- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 msgr2=0x7f696015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-16T17:57:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965200640 1 -- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 msgr2=0x7f696015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T17:57:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965200640 1 --2- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 0x7f696015e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T17:57:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965200640 1 --2- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 0x7f696015e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T17:57:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965c00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69600ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965c00640 1 -- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 msgr2=0x7f69600ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-16T17:57:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965c00640 1 -- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 msgr2=0x7f69600ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T17:57:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965c00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69600ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T17:57:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:20.906+0000 7f6965c00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69600ce9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T17:57:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.107+0000 7f6965200640 1 --2- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 0x7f696015e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.107+0000 7f6965c00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69600ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.107+0000 7f6965200640 1 -- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 msgr2=0x7f696015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-16T17:57:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.107+0000 7f6965200640 1 -- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 msgr2=0x7f696015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T17:57:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.107+0000 7f6965200640 1 --2- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 0x7f696015e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T17:57:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.108+0000 7f6965200640 1 --2- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 0x7f696015e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T17:57:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.108+0000 7f6965c00640 1 -- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 msgr2=0x7f69600ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-16T17:57:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.108+0000 7f6965c00640 1 -- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 msgr2=0x7f69600ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T17:57:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.108+0000 7f6965c00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69600ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T17:57:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.108+0000 7f6965c00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69600ce9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T17:57:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.160+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806178175 0 0) 0x7f69600b33c0 con 0x7f69600d48e0 2024-03-16T17:57:21.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.160+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6944003520 con 0x7f69600d48e0 2024-03-16T17:57:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.168+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 <== mon.1 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4205340103 0 0) 0x7f6944003520 con 0x7f69600d48e0 2024-03-16T17:57:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.168+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 msgr2=0x7f69600ce9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T17:57:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.168+0000 7f695f400640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69600ce9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.168+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 msgr2=0x7f696015e420 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T17:57:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.168+0000 7f695f400640 1 --2- 172.21.15.82:0/630638991 >> v2:172.21.15.82:3300/0 conn(0x7f696015e040 0x7f696015e420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.168+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69600b3610 con 0x7f69600d48e0 2024-03-16T17:57:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.168+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 <== mon.1 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f694c001670 con 0x7f69600d48e0 2024-03-16T17:57:21.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.168+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 >> v1:172.21.15.90:6789/0 conn(0x7f69600d48e0 legacy=0x7f69600d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.168+0000 7f695f400640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944003970 0x7f6944003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.168+0000 7f695f400640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69440047c0 0x7f6944004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.169+0000 7f695f400640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69440051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.169+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600d1cd0 con 0x7f69440047c0 2024-03-16T17:57:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.169+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600d1fb0 con 0x7f6944003970 2024-03-16T17:57:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.169+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f694400bb40 con 0x7f69600d2ab0 2024-03-16T17:57:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.169+0000 7f695fe00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69440047c0 0x7f6944004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.169+0000 7f6965c00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69440051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.169+0000 7f6965200640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944003970 0x7f6944003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:21.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.170+0000 7f695fe00640 1 -- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 msgr2=0x7f69440051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.170+0000 7f695fe00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69600d2ab0 0x7f69440051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.170+0000 7f695fe00640 1 -- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944003970 msgr2=0x7f6944003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.170+0000 7f695fe00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6944003970 0x7f6944003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.170+0000 7f695fe00640 1 -- 172.21.15.82:0/630638991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6944003520 con 0x7f69440047c0 2024-03-16T17:57:21.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.172+0000 7f695fe00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69440047c0 0x7f6944004c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f694c007f20 tx=0x7f69440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedfe3d0be4fa4fd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:21.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.173+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c02c940 con 0x7f69440047c0 2024-03-16T17:57:21.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.173+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c02caa0 con 0x7f69440047c0 2024-03-16T17:57:21.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.173+0000 7f695f400640 1 -- 172.21.15.82:0/630638991 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c02ce50 con 0x7f69440047c0 2024-03-16T17:57:21.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.174+0000 7f6967c00640 1 -- 172.21.15.82:0/630638991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69440047c0 msgr2=0x7f6944004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:21.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.175+0000 7f6967c00640 1 --2- 172.21.15.82:0/630638991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69440047c0 0x7f6944004c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f694c007f20 tx=0x7f69440057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:21.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.175+0000 7f6965c00640 1 -- 172.21.15.82:0/630638991 reap_dead start 2024-03-16T17:57:21.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.175+0000 7f6967c00640 1 -- 172.21.15.82:0/630638991 shutdown_connections 2024-03-16T17:57:21.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.175+0000 7f6967c00640 1 -- 172.21.15.82:0/630638991 >> 172.21.15.82:0/630638991 conn(0x7f69600d3dc0 msgr2=0x7f69600d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:21.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.175+0000 7f6967c00640 1 -- 172.21.15.82:0/630638991 shutdown_connections 2024-03-16T17:57:21.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.175+0000 7f6967c00640 1 -- 172.21.15.82:0/630638991 wait complete. 2024-03-16T17:57:21.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.178+0000 7f6967c00640 1 Processor -- start 2024-03-16T17:57:21.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.178+0000 7f6967c00640 1 -- start start 2024-03-16T17:57:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f6967c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69440047c0 0x7f696006c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f6967c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69600cb1a0 0x7f69600cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f6967c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69600cd170 0x7f69600cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f6967c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600cc1f0 con 0x7f69600cb1a0 2024-03-16T17:57:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f6967c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69600cbf10 con 0x7f69600cd170 2024-03-16T17:57:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f6967c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69600cc690 con 0x7f69440047c0 2024-03-16T17:57:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f6965200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69440047c0 0x7f696006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:21.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f695fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69600cb1a0 0x7f69600cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:21.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f6965200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69440047c0 0x7f696006c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:37962/0 (socket says 172.21.15.82:37962) 2024-03-16T17:57:21.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f6965200640 1 -- 172.21.15.82:0/2540167710 learned_addr learned my addr 172.21.15.82:0/2540167710 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:21.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.180+0000 7f6965c00640 1 --2- 172.21.15.82:0/2540167710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69600cd170 0x7f69600cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:21.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.181+0000 7f6965200640 1 -- 172.21.15.82:0/2540167710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69600cd170 msgr2=0x7f69600cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.181+0000 7f6965200640 1 --2- 172.21.15.82:0/2540167710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69600cd170 0x7f69600cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.181+0000 7f6965200640 1 -- 172.21.15.82:0/2540167710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69600cb1a0 msgr2=0x7f69600cb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.181+0000 7f6965200640 1 --2- 172.21.15.82:0/2540167710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69600cb1a0 0x7f69600cb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.181+0000 7f6965200640 1 -- 172.21.15.82:0/2540167710 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69600cc970 con 0x7f69440047c0 2024-03-16T17:57:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.181+0000 7f695fe00640 1 --2- 172.21.15.82:0/2540167710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69600cb1a0 0x7f69600cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.181+0000 7f6965200640 1 --2- 172.21.15.82:0/2540167710 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69440047c0 0x7f696006c780 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f69500136a0 tx=0x7f6950013b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.183+0000 7f695cc00640 1 -- 172.21.15.82:0/2540167710 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950004440 con 0x7f69440047c0 2024-03-16T17:57:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.183+0000 7f695cc00640 1 -- 172.21.15.82:0/2540167710 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695000c070 con 0x7f69440047c0 2024-03-16T17:57:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.183+0000 7f6967c00640 1 -- 172.21.15.82:0/2540167710 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69600ccc50 con 0x7f69440047c0 2024-03-16T17:57:21.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.183+0000 7f695cc00640 1 -- 172.21.15.82:0/2540167710 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950011a60 con 0x7f69440047c0 2024-03-16T17:57:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.183+0000 7f6967c00640 1 -- 172.21.15.82:0/2540167710 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696006ccc0 con 0x7f69440047c0 2024-03-16T17:57:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.183+0000 7f695cc00640 1 -- 172.21.15.82:0/2540167710 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6950011bc0 con 0x7f69440047c0 2024-03-16T17:57:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.183+0000 7f695cc00640 1 -- 172.21.15.82:0/2540167710 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6950003d10 con 0x7f69440047c0 2024-03-16T17:57:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.184+0000 7f6967c00640 1 -- 172.21.15.82:0/2540167710 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6960067f30 con 0x7f69440047c0 2024-03-16T17:57:21.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.188+0000 7f695cc00640 1 -- 172.21.15.82:0/2540167710 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6950024060 con 0x7f69440047c0 2024-03-16T17:57:21.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.261+0000 7f6967c00640 1 -- 172.21.15.82:0/2540167710 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6960074840 con 0x7f69440047c0 2024-03-16T17:57:21.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.272+0000 7f695cc00640 1 -- 172.21.15.82:0/2540167710 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f695001a070 con 0x7f69440047c0 2024-03-16T17:57:21.276 INFO:teuthology.orchestra.run.smithi082.stderr:adjusted tunables profile to default 2024-03-16T17:57:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.273+0000 7f6967c00640 1 -- 172.21.15.82:0/2540167710 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69440047c0 msgr2=0x7f696006c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.273+0000 7f6967c00640 1 --2- 172.21.15.82:0/2540167710 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69440047c0 0x7f696006c780 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f69500136a0 tx=0x7f6950013b60 comp rx=0 tx=0).stop 2024-03-16T17:57:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.273+0000 7f6967c00640 1 -- 172.21.15.82:0/2540167710 shutdown_connections 2024-03-16T17:57:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.273+0000 7f6967c00640 1 --2- 172.21.15.82:0/2540167710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69600cd170 0x7f69600cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.273+0000 7f6967c00640 1 --2- 172.21.15.82:0/2540167710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69600cb1a0 0x7f69600cb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.273+0000 7f6967c00640 1 --2- 172.21.15.82:0/2540167710 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f69440047c0 0x7f696006c780 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.273+0000 7f6967c00640 1 -- 172.21.15.82:0/2540167710 >> 172.21.15.82:0/2540167710 conn(0x7f69600d3dc0 msgr2=0x7f69600b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.274+0000 7f6967c00640 1 -- 172.21.15.82:0/2540167710 shutdown_connections 2024-03-16T17:57:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:21.274+0000 7f6967c00640 1 -- 172.21.15.82:0/2540167710 wait complete. 2024-03-16T17:57:21.293 INFO:tasks.ceph:check_enable_crimson: False 2024-03-16T17:57:21.294 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-16T17:57:21.294 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:21.294 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-16T17:57:21.374 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:21.374 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-16T17:57:21.504 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:21.504 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-16T17:57:21.631 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:57:21.631 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-16T17:57:21.757 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:21.757 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-16T17:57:21.759 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:21.756+0000 7f3dda63f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T17:57:21.765 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:21.755+0000 7f798dcda040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T17:57:21.822 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:21.819+0000 7f3dda63f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T17:57:21.826 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:21.823+0000 7f798dcda040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T17:57:21.845 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:21.845 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-16T17:57:21.978 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:21.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-16T17:57:22.102 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:57:22.102 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-16T17:57:22.220 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new 3e9a1199-4782-4263-98ef-1885f932bc89 0 2024-03-16T17:57:22.435 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:22.431+0000 7f3dda63f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T17:57:22.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.444+0000 7f3ad3e00640 1 Processor -- start 2024-03-16T17:57:22.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.444+0000 7f3ad3e00640 1 -- start start 2024-03-16T17:57:22.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.445+0000 7f3ad3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3ad40cf9e0 0x7f3ad40d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:22.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.445+0000 7f3ad3e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad40b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:22.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad3e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad40b3910 con 0x7f3ad415e000 2024-03-16T17:57:22.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad3e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad415e3e0 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad3e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40d2580 con 0x7f3ad40d0ca0 2024-03-16T17:57:22.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad3400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad40b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:22.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad2000640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3ad40cf9e0 0x7f3ad40d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:22.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad3400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad40b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:33856/0 (socket says 172.21.15.90:33856) 2024-03-16T17:57:22.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad3400640 1 -- 172.21.15.90:0/1644843515 learned_addr learned my addr 172.21.15.90:0/1644843515 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:22.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad3400640 1 -- 172.21.15.90:0/1644843515 >> v2:172.21.15.90:3300/0 conn(0x7f3ad40cf9e0 msgr2=0x7f3ad40d46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:22.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad3400640 1 --2- 172.21.15.90:0/1644843515 >> v2:172.21.15.90:3300/0 conn(0x7f3ad40cf9e0 0x7f3ad40d46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad3400640 1 -- 172.21.15.90:0/1644843515 >> v1:172.21.15.82:6789/0 conn(0x7f3ad415e000 legacy=0x7f3ad40d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:22.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad3400640 1 -- 172.21.15.90:0/1644843515 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad40d2860 con 0x7f3ad40d0ca0 2024-03-16T17:57:22.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.446+0000 7f3ad2000640 1 --2- 172.21.15.90:0/1644843515 >> v2:172.21.15.90:3300/0 conn(0x7f3ad40cf9e0 0x7f3ad40d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:22.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.447+0000 7f3ad3400640 1 --2- 172.21.15.90:0/1644843515 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad40b3da0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009e50 tx=0x7f3acc02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73d0cefdd4498e77 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:22.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad1600640 1 -- 172.21.15.90:0/1644843515 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc02cb00 con 0x7f3ad40d0ca0 2024-03-16T17:57:22.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad1600640 1 -- 172.21.15.90:0/1644843515 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc02cc60 con 0x7f3ad40d0ca0 2024-03-16T17:57:22.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad1600640 1 -- 172.21.15.90:0/1644843515 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0343f0 con 0x7f3ad40d0ca0 2024-03-16T17:57:22.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad3e00640 1 -- 172.21.15.90:0/1644843515 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 msgr2=0x7f3ad40b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:22.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad3e00640 1 --2- 172.21.15.90:0/1644843515 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad40b3da0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009e50 tx=0x7f3acc02bf60 comp rx=0 tx=0).stop 2024-03-16T17:57:22.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad3e00640 1 -- 172.21.15.90:0/1644843515 shutdown_connections 2024-03-16T17:57:22.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad3e00640 1 --2- 172.21.15.90:0/1644843515 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad40b3da0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad3e00640 1 --2- 172.21.15.90:0/1644843515 >> v2:172.21.15.90:3300/0 conn(0x7f3ad40cf9e0 0x7f3ad40d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad3e00640 1 -- 172.21.15.90:0/1644843515 >> 172.21.15.90:0/1644843515 conn(0x7f3ad40d3bc0 msgr2=0x7f3ad40d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:22.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad3e00640 1 -- 172.21.15.90:0/1644843515 shutdown_connections 2024-03-16T17:57:22.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.448+0000 7f3ad3e00640 1 -- 172.21.15.90:0/1644843515 wait complete. 2024-03-16T17:57:22.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.450+0000 7f3ad3e00640 1 Processor -- start 2024-03-16T17:57:22.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.450+0000 7f3ad3e00640 1 -- start start 2024-03-16T17:57:22.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ad40cf9e0 0x7f3ad4156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:22.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad3e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad406c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:22.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad415e000 0x7f3ad406c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:22.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40b8ed0 con 0x7f3ad415e000 2024-03-16T17:57:22.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad3e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40cb870 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad3e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40cbb50 con 0x7f3ad40d0ca0 2024-03-16T17:57:22.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ad40cf9e0 0x7f3ad4156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:22.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad2000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad406c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:22.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ad40cf9e0 0x7f3ad4156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56430/0 (socket says 172.21.15.90:56430) 2024-03-16T17:57:22.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad2a00640 1 -- 172.21.15.90:0/3437638843 learned_addr learned my addr 172.21.15.90:0/3437638843 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:22.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad2000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad406c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:33868/0 (socket says 172.21.15.90:33868) 2024-03-16T17:57:22.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.451+0000 7f3ad3400640 1 --2- 172.21.15.90:0/3437638843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad415e000 0x7f3ad406c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:22.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3ad2a00640 1 -- 172.21.15.90:0/3437638843 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 msgr2=0x7f3ad406c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:22.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3ad2a00640 1 --2- 172.21.15.90:0/3437638843 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad406c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3ad2a00640 1 -- 172.21.15.90:0/3437638843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad415e000 msgr2=0x7f3ad406c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:22.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3ad2a00640 1 --2- 172.21.15.90:0/3437638843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad415e000 0x7f3ad406c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3ad2a00640 1 -- 172.21.15.90:0/3437638843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac0009400 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3ad2a00640 1 --2- 172.21.15.90:0/3437638843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ad40cf9e0 0x7f3ad4156310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400c400 tx=0x7f3ac400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:22.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3acaa00640 1 -- 172.21.15.90:0/3437638843 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac40052d0 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3acaa00640 1 -- 172.21.15.90:0/3437638843 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac4005430 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3ad3e00640 1 -- 172.21.15.90:0/3437638843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3acc009b30 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3acaa00640 1 -- 172.21.15.90:0/3437638843 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4003830 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3ad3e00640 1 -- 172.21.15.90:0/3437638843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad409de40 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.452+0000 7f3acaa00640 1 -- 172.21.15.90:0/3437638843 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3ac4003990 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.453+0000 7f3acaa00640 1 -- 172.21.15.90:0/3437638843 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3ac401c780 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.453+0000 7f3ad3e00640 1 -- 172.21.15.90:0/3437638843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad40cc070 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.456+0000 7f3acaa00640 1 -- 172.21.15.90:0/3437638843 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3ad40cc070 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.470 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:22.460+0000 7f798dcda040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T17:57:22.499 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:22.496+0000 7f3dda63f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T17:57:22.533 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:22.529+0000 7f798dcda040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T17:57:22.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.535+0000 7f3ad3e00640 1 -- 172.21.15.90:0/3437638843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3e9a1199-4782-4263-98ef-1885f932bc89", "id": 0} v 0) v1 -- 0x7f3ad409dc80 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.541+0000 7f3acaa00640 1 -- 172.21.15.90:0/3437638843 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3e9a1199-4782-4263-98ef-1885f932bc89", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3ac4017070 con 0x7f3ad40cf9e0 2024-03-16T17:57:22.545 INFO:teuthology.orchestra.run.smithi090.stdout:0 2024-03-16T17:57:22.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.543+0000 7f3ad3e00640 1 -- 172.21.15.90:0/3437638843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ad40cf9e0 msgr2=0x7f3ad4156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:22.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.543+0000 7f3ad3e00640 1 --2- 172.21.15.90:0/3437638843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ad40cf9e0 0x7f3ad4156310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400c400 tx=0x7f3ac400c8c0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.543+0000 7f3ad3e00640 1 -- 172.21.15.90:0/3437638843 shutdown_connections 2024-03-16T17:57:22.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.543+0000 7f3ad3e00640 1 --2- 172.21.15.90:0/3437638843 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ad415e000 0x7f3ad406c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.543+0000 7f3ad3e00640 1 --2- 172.21.15.90:0/3437638843 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3ad40d0ca0 0x7f3ad406c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.543+0000 7f3ad3e00640 1 --2- 172.21.15.90:0/3437638843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3ad40cf9e0 0x7f3ad4156310 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.543+0000 7f3ad3e00640 1 -- 172.21.15.90:0/3437638843 >> 172.21.15.90:0/3437638843 conn(0x7f3ad40d3bc0 msgr2=0x7f3ad40b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:22.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.543+0000 7f3ad3e00640 1 -- 172.21.15.90:0/3437638843 shutdown_connections 2024-03-16T17:57:22.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.543+0000 7f3ad3e00640 1 -- 172.21.15.90:0/3437638843 wait complete. 2024-03-16T17:57:22.550 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:22.547+0000 7f3dda63f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T17:57:22.565 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new 57f8eb0b-64d2-4879-96ce-c786ff39aaf0 1 2024-03-16T17:57:22.591 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:22.587+0000 7f798dcda040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T17:57:22.612 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:22.609+0000 7f3dda63f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T17:57:22.660 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:22.656+0000 7f798dcda040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T17:57:22.662 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:22.658+0000 7f3dda63f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T17:57:22.713 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:22.709+0000 7f798dcda040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T17:57:22.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.749+0000 7f1ccd600640 1 Processor -- start 2024-03-16T17:57:22.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.749+0000 7f1ccd600640 1 -- start start 2024-03-16T17:57:22.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.751+0000 7f1ccd600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1cc815df00 0x7f1cc80d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:22.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.751+0000 7f1ccd600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc80b32c0 0x7f1cc80d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:22.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.751+0000 7f1ccd600640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80cfe40 con 0x7f1cc815e2e0 2024-03-16T17:57:22.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.751+0000 7f1ccd600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc80d4940 con 0x7f1cc815df00 2024-03-16T17:57:22.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.751+0000 7f1ccd600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc80b3780 con 0x7f1cc80b32c0 2024-03-16T17:57:22.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.751+0000 7f1cc7400640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f1cc815e2e0 0x7f1cc80ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:42120/0 (socket says 172.21.15.90:42120) 2024-03-16T17:57:22.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.751+0000 7f1cc6000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc80b32c0 0x7f1cc80d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:22.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.751+0000 7f1cc7400640 1 -- 172.21.15.90:0/509332200 learned_addr learned my addr 172.21.15.90:0/509332200 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-16T17:57:22.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.751+0000 7f1cc6a00640 1 --2- 172.21.15.90:0/509332200 >> v2:172.21.15.82:3300/0 conn(0x7f1cc815df00 0x7f1cc80d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:22.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.752+0000 7f1cc6000640 1 -- 172.21.15.90:0/509332200 >> v2:172.21.15.82:3300/0 conn(0x7f1cc815df00 msgr2=0x7f1cc80d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:22.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.752+0000 7f1cc6000640 1 --2- 172.21.15.90:0/509332200 >> v2:172.21.15.82:3300/0 conn(0x7f1cc815df00 0x7f1cc80d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.752+0000 7f1cc5600640 1 -- 172.21.15.90:0/509332200 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657899005 0 0) 0x7f1cc80cfe40 con 0x7f1cc815e2e0 2024-03-16T17:57:22.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.752+0000 7f1cc6000640 1 -- 172.21.15.90:0/509332200 >> v1:172.21.15.90:6789/0 conn(0x7f1cc815e2e0 legacy=0x7f1cc80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:22.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.752+0000 7f1cc6000640 1 -- 172.21.15.90:0/509332200 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc80b3a60 con 0x7f1cc80b32c0 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.753+0000 7f1cc6000640 1 --2- 172.21.15.90:0/509332200 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc80b32c0 0x7f1cc80d2360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40097a0 tx=0x7f1cb402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bdef9d0084afa8c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.753+0000 7f1cc5600640 1 -- 172.21.15.90:0/509332200 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4030610 con 0x7f1cc80b32c0 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.753+0000 7f1cc5600640 1 -- 172.21.15.90:0/509332200 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb4030770 con 0x7f1cc80b32c0 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.753+0000 7f1cc5600640 1 -- 172.21.15.90:0/509332200 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4030b00 con 0x7f1cc80b32c0 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.754+0000 7f1ccd600640 1 -- 172.21.15.90:0/509332200 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc80b32c0 msgr2=0x7f1cc80d2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.754+0000 7f1ccd600640 1 --2- 172.21.15.90:0/509332200 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc80b32c0 0x7f1cc80d2360 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40097a0 tx=0x7f1cb402b960 comp rx=0 tx=0).stop 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.754+0000 7f1ccd600640 1 -- 172.21.15.90:0/509332200 shutdown_connections 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.754+0000 7f1ccd600640 1 --2- 172.21.15.90:0/509332200 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc80b32c0 0x7f1cc80d2360 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.754+0000 7f1ccd600640 1 --2- 172.21.15.90:0/509332200 >> v2:172.21.15.82:3300/0 conn(0x7f1cc815df00 0x7f1cc80d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.754+0000 7f1ccd600640 1 -- 172.21.15.90:0/509332200 >> 172.21.15.90:0/509332200 conn(0x7f1cc80d3d80 msgr2=0x7f1cc80d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.754+0000 7f1ccd600640 1 -- 172.21.15.90:0/509332200 shutdown_connections 2024-03-16T17:57:22.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.754+0000 7f1ccd600640 1 -- 172.21.15.90:0/509332200 wait complete. 2024-03-16T17:57:22.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.755+0000 7f1ccd600640 1 Processor -- start 2024-03-16T17:57:22.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.755+0000 7f1ccd600640 1 -- start start 2024-03-16T17:57:22.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.756+0000 7f1ccd600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc80b32c0 0x7f1cc815f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:22.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.756+0000 7f1ccd600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cc815df00 0x7f1cc81f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:22.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.756+0000 7f1ccd600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc815e2e0 0x7f1cc80cf240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:22.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.756+0000 7f1ccd600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc815ec30 con 0x7f1cc815df00 2024-03-16T17:57:22.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.756+0000 7f1ccd600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc815f1f0 con 0x7f1cc80b32c0 2024-03-16T17:57:22.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.756+0000 7f1ccd600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc81f43f0 con 0x7f1cc815e2e0 2024-03-16T17:57:22.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc80b32c0 0x7f1cc815f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:22.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc7400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc815e2e0 0x7f1cc80cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc80b32c0 0x7f1cc815f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56446/0 (socket says 172.21.15.90:56446) 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6a00640 1 -- 172.21.15.90:0/1259059396 learned_addr learned my addr 172.21.15.90:0/1259059396 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6000640 1 --2- 172.21.15.90:0/1259059396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cc815df00 0x7f1cc81f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6a00640 1 -- 172.21.15.90:0/1259059396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc815e2e0 msgr2=0x7f1cc80cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6a00640 1 --2- 172.21.15.90:0/1259059396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc815e2e0 0x7f1cc80cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6a00640 1 -- 172.21.15.90:0/1259059396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cc815df00 msgr2=0x7f1cc81f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6a00640 1 --2- 172.21.15.90:0/1259059396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cc815df00 0x7f1cc81f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6a00640 1 -- 172.21.15.90:0/1259059396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4009400 con 0x7f1cc80b32c0 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6000640 1 --2- 172.21.15.90:0/1259059396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cc815df00 0x7f1cc81f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc7400640 1 --2- 172.21.15.90:0/1259059396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc815e2e0 0x7f1cc80cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.757+0000 7f1cc6a00640 1 --2- 172.21.15.90:0/1259059396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc80b32c0 0x7f1cc815f680 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800d800 tx=0x7f1cb800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.758+0000 7f1cbea00640 1 -- 172.21.15.90:0/1259059396 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800b6c0 con 0x7f1cc80b32c0 2024-03-16T17:57:22.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.758+0000 7f1cbea00640 1 -- 172.21.15.90:0/1259059396 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb800b820 con 0x7f1cc80b32c0 2024-03-16T17:57:22.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.758+0000 7f1ccd600640 1 -- 172.21.15.90:0/1259059396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc81f46d0 con 0x7f1cc80b32c0 2024-03-16T17:57:22.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.758+0000 7f1cbea00640 1 -- 172.21.15.90:0/1259059396 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800bbb0 con 0x7f1cc80b32c0 2024-03-16T17:57:22.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.758+0000 7f1ccd600640 1 -- 172.21.15.90:0/1259059396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc80b1fb0 con 0x7f1cc80b32c0 2024-03-16T17:57:22.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.758+0000 7f1cbea00640 1 -- 172.21.15.90:0/1259059396 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1cb800bd10 con 0x7f1cc80b32c0 2024-03-16T17:57:22.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.758+0000 7f1cbea00640 1 -- 172.21.15.90:0/1259059396 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f1cb80149c0 con 0x7f1cc80b32c0 2024-03-16T17:57:22.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.759+0000 7f1ccd600640 1 -- 172.21.15.90:0/1259059396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc80b2140 con 0x7f1cc80b32c0 2024-03-16T17:57:22.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.761+0000 7f1cbea00640 1 -- 172.21.15.90:0/1259059396 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1cb8014b80 con 0x7f1cc80b32c0 2024-03-16T17:57:22.791 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:22.788+0000 7f3dda63f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T17:57:22.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.840+0000 7f1ccd600640 1 -- 172.21.15.90:0/1259059396 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd new", "uuid": "57f8eb0b-64d2-4879-96ce-c786ff39aaf0", "id": 1} v 0) v1 -- 0x7f1cc81f48c0 con 0x7f1cc80b32c0 2024-03-16T17:57:22.850 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:22.847+0000 7f798dcda040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-16T17:57:22.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.848+0000 7f1cbea00640 1 -- 172.21.15.90:0/1259059396 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "57f8eb0b-64d2-4879-96ce-c786ff39aaf0", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1cb8014e50 con 0x7f1cc80b32c0 2024-03-16T17:57:22.852 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-03-16T17:57:22.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.850+0000 7f1ccd600640 1 -- 172.21.15.90:0/1259059396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc80b32c0 msgr2=0x7f1cc815f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:22.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.850+0000 7f1ccd600640 1 --2- 172.21.15.90:0/1259059396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc80b32c0 0x7f1cc815f680 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800d800 tx=0x7f1cb800dcc0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.850+0000 7f1ccd600640 1 -- 172.21.15.90:0/1259059396 shutdown_connections 2024-03-16T17:57:22.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.850+0000 7f1ccd600640 1 --2- 172.21.15.90:0/1259059396 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1cc815e2e0 0x7f1cc80cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.850+0000 7f1ccd600640 1 --2- 172.21.15.90:0/1259059396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cc815df00 0x7f1cc81f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.850+0000 7f1ccd600640 1 --2- 172.21.15.90:0/1259059396 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1cc80b32c0 0x7f1cc815f680 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:22.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.850+0000 7f1ccd600640 1 -- 172.21.15.90:0/1259059396 >> 172.21.15.90:0/1259059396 conn(0x7f1cc80d3d80 msgr2=0x7f1cc80d2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:22.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.850+0000 7f1ccd600640 1 -- 172.21.15.90:0/1259059396 shutdown_connections 2024-03-16T17:57:22.854 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:22.850+0000 7f1ccd600640 1 -- 172.21.15.90:0/1259059396 wait complete. 2024-03-16T17:57:22.875 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new 93d45d8a-6281-41da-a4dd-bcbb25bb24a6 2 2024-03-16T17:57:22.876 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:22.868+0000 7f3dda63f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T17:57:22.924 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:22.921+0000 7f3dda63f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T17:57:22.935 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:22.932+0000 7f798dcda040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T17:57:22.985 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:22.981+0000 7f3dda63f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T17:57:22.992 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:22.988+0000 7f798dcda040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T17:57:23.039 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:23.036+0000 7f3dda63f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T17:57:23.056 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:23.052+0000 7f798dcda040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T17:57:23.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.104+0000 7f5f3cc00640 1 Processor -- start 2024-03-16T17:57:23.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.104+0000 7f5f3cc00640 1 -- start start 2024-03-16T17:57:23.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.105+0000 7f5f3cc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f380cf9e0 0x7f5f380d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.105+0000 7f5f3cc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380b3910 con 0x7f5f3815e000 2024-03-16T17:57:23.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.105+0000 7f5f3cc00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f380d2580 con 0x7f5f380d0ca0 2024-03-16T17:57:23.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.105+0000 7f5f3cc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3815e3e0 con 0x7f5f380cf9e0 2024-03-16T17:57:23.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.106+0000 7f5f37e00640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f5f380d0ca0 0x7f5f380b3da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:42132/0 (socket says 172.21.15.90:42132) 2024-03-16T17:57:23.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.106+0000 7f5f37e00640 1 -- 172.21.15.90:0/3221017751 learned_addr learned my addr 172.21.15.90:0/3221017751 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-16T17:57:23.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.106+0000 7f5f36a00640 1 --2- 172.21.15.90:0/3221017751 >> v2:172.21.15.82:3300/0 conn(0x7f5f380cf9e0 0x7f5f380d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.106+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763637384 0 0) 0x7f5f380d2580 con 0x7f5f380d0ca0 2024-03-16T17:57:23.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.106+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f1c003520 con 0x7f5f380d0ca0 2024-03-16T17:57:23.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.106+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 <== mon.1 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1798580615 0 0) 0x7f5f1c003520 con 0x7f5f380d0ca0 2024-03-16T17:57:23.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.106+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 >> v2:172.21.15.82:3300/0 conn(0x7f5f380cf9e0 msgr2=0x7f5f380d46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:23.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.106+0000 7f5f36000640 1 --2- 172.21.15.90:0/3221017751 >> v2:172.21.15.82:3300/0 conn(0x7f5f380cf9e0 0x7f5f380d46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 >> v1:172.21.15.82:6789/0 conn(0x7f5f3815e000 legacy=0x7f5f380d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 --> v1:172.21.15.90:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f380d2860 con 0x7f5f380d0ca0 2024-03-16T17:57:23.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 <== mon.1 v1:172.21.15.90:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f5f300019f0 con 0x7f5f380d0ca0 2024-03-16T17:57:23.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 >> v1:172.21.15.90:6789/0 conn(0x7f5f380d0ca0 legacy=0x7f5f380b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36000640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 0x7f5f1c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36000640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f1c0047c0 0x7f5f1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36000640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f1c005180 0x7f5f1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c00c080 con 0x7f5f1c0047c0 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c00c1f0 con 0x7f5f1c0039e0 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c00c3b0 con 0x7f5f1c005180 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36a00640 1 --2- 172.21.15.90:0/3221017751 >> v2:172.21.15.82:3300/0 conn(0x7f5f380cf9e0 0x7f5f380d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f37400640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 0x7f5f1c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f37e00640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f1c005180 0x7f5f1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f36a00640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f1c0047c0 0x7f5f1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f37400640 1 -- 172.21.15.90:0/3221017751 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f1c005180 msgr2=0x7f5f1c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f37400640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f1c005180 0x7f5f1c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f37400640 1 -- 172.21.15.90:0/3221017751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f1c0047c0 msgr2=0x7f5f1c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f37400640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f1c0047c0 0x7f5f1c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.107+0000 7f5f37400640 1 -- 172.21.15.90:0/3221017751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c003520 con 0x7f5f1c0039e0 2024-03-16T17:57:23.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.109+0000 7f5f37e00640 1 -- 172.21.15.90:0/3221017751 reap_dead start 2024-03-16T17:57:23.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.109+0000 7f5f37400640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 0x7f5f1c003e00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f28001750 tx=0x7f5f1c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=760f1ef85d39dead server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:23.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.109+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280052d0 con 0x7f5f1c0039e0 2024-03-16T17:57:23.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.109+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2802e050 con 0x7f5f1c0039e0 2024-03-16T17:57:23.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.109+0000 7f5f36000640 1 -- 172.21.15.90:0/3221017751 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280035f0 con 0x7f5f1c0039e0 2024-03-16T17:57:23.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.110+0000 7f5f3cc00640 1 -- 172.21.15.90:0/3221017751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 msgr2=0x7f5f1c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.110+0000 7f5f3cc00640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 0x7f5f1c003e00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5f28001750 tx=0x7f5f1c0057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.110+0000 7f5f3cc00640 1 -- 172.21.15.90:0/3221017751 shutdown_connections 2024-03-16T17:57:23.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.110+0000 7f5f3cc00640 1 --2- 172.21.15.90:0/3221017751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 0x7f5f1c003e00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.110+0000 7f5f3cc00640 1 -- 172.21.15.90:0/3221017751 >> 172.21.15.90:0/3221017751 conn(0x7f5f380d3bc0 msgr2=0x7f5f380d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:23.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.110+0000 7f5f3cc00640 1 -- 172.21.15.90:0/3221017751 shutdown_connections 2024-03-16T17:57:23.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.110+0000 7f5f3cc00640 1 -- 172.21.15.90:0/3221017751 wait complete. 2024-03-16T17:57:23.116 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:23.112+0000 7f798dcda040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T17:57:23.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.112+0000 7f5f3cc00640 1 Processor -- start 2024-03-16T17:57:23.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.112+0000 7f5f3cc00640 1 -- start start 2024-03-16T17:57:23.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f3cc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 0x7f5f3815be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f3cc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f380b19d0 0x7f5f380b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f3cc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f380b7d20 0x7f5f380b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f3cc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f380b2390 con 0x7f5f380b7d20 2024-03-16T17:57:23.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f3cc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3815c3b0 con 0x7f5f1c0039e0 2024-03-16T17:57:23.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f3cc00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f381601d0 con 0x7f5f380b19d0 2024-03-16T17:57:23.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f36a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f380b19d0 0x7f5f380b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f37400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 0x7f5f3815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f36a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f380b19d0 0x7f5f380b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:33902/0 (socket says 172.21.15.90:33902) 2024-03-16T17:57:23.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f36a00640 1 -- 172.21.15.90:0/2638436568 learned_addr learned my addr 172.21.15.90:0/2638436568 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:23.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f37e00640 1 --2- 172.21.15.90:0/2638436568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f380b7d20 0x7f5f380b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f37400640 1 -- 172.21.15.90:0/2638436568 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f380b19d0 msgr2=0x7f5f380b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f37400640 1 --2- 172.21.15.90:0/2638436568 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f380b19d0 0x7f5f380b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f37400640 1 -- 172.21.15.90:0/2638436568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f380b7d20 msgr2=0x7f5f380b81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.113+0000 7f5f37400640 1 --2- 172.21.15.90:0/2638436568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f380b7d20 0x7f5f380b81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.114+0000 7f5f37400640 1 -- 172.21.15.90:0/2638436568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f38160420 con 0x7f5f1c0039e0 2024-03-16T17:57:23.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.114+0000 7f5f36a00640 1 --2- 172.21.15.90:0/2638436568 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f380b19d0 0x7f5f380b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:23.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.114+0000 7f5f37400640 1 --2- 172.21.15.90:0/2638436568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 0x7f5f3815be70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5f28004000 tx=0x7f5f28001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:23.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.114+0000 7f5f2f400640 1 -- 172.21.15.90:0/2638436568 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f280079f0 con 0x7f5f1c0039e0 2024-03-16T17:57:23.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.114+0000 7f5f2f400640 1 -- 172.21.15.90:0/2638436568 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f28007640 con 0x7f5f1c0039e0 2024-03-16T17:57:23.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.114+0000 7f5f2f400640 1 -- 172.21.15.90:0/2638436568 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f28006c20 con 0x7f5f1c0039e0 2024-03-16T17:57:23.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.114+0000 7f5f3cc00640 1 -- 172.21.15.90:0/2638436568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f38160700 con 0x7f5f1c0039e0 2024-03-16T17:57:23.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.114+0000 7f5f3cc00640 1 -- 172.21.15.90:0/2638436568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f38160cc0 con 0x7f5f1c0039e0 2024-03-16T17:57:23.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.115+0000 7f5f2f400640 1 -- 172.21.15.90:0/2638436568 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5f28003980 con 0x7f5f1c0039e0 2024-03-16T17:57:23.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.115+0000 7f5f2f400640 1 -- 172.21.15.90:0/2638436568 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f5f2803dde0 con 0x7f5f1c0039e0 2024-03-16T17:57:23.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.115+0000 7f5f3cc00640 1 -- 172.21.15.90:0/2638436568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f38160e50 con 0x7f5f1c0039e0 2024-03-16T17:57:23.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.117+0000 7f5f2f400640 1 -- 172.21.15.90:0/2638436568 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5f28041020 con 0x7f5f1c0039e0 2024-03-16T17:57:23.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.198+0000 7f5f3cc00640 1 -- 172.21.15.90:0/2638436568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd new", "uuid": "93d45d8a-6281-41da-a4dd-bcbb25bb24a6", "id": 2} v 0) v1 -- 0x7f5f3804f250 con 0x7f5f1c0039e0 2024-03-16T17:57:23.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.206+0000 7f5f2f400640 1 -- 172.21.15.90:0/2638436568 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "93d45d8a-6281-41da-a4dd-bcbb25bb24a6", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5f28001370 con 0x7f5f1c0039e0 2024-03-16T17:57:23.210 INFO:teuthology.orchestra.run.smithi090.stdout:2 2024-03-16T17:57:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.207+0000 7f5f3cc00640 1 -- 172.21.15.90:0/2638436568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 msgr2=0x7f5f3815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.207+0000 7f5f3cc00640 1 --2- 172.21.15.90:0/2638436568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 0x7f5f3815be70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5f28004000 tx=0x7f5f28001120 comp rx=0 tx=0).stop 2024-03-16T17:57:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.207+0000 7f5f3cc00640 1 -- 172.21.15.90:0/2638436568 shutdown_connections 2024-03-16T17:57:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.207+0000 7f5f3cc00640 1 --2- 172.21.15.90:0/2638436568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f380b7d20 0x7f5f380b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.207+0000 7f5f3cc00640 1 --2- 172.21.15.90:0/2638436568 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5f380b19d0 0x7f5f380b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.207+0000 7f5f3cc00640 1 --2- 172.21.15.90:0/2638436568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f1c0039e0 0x7f5f3815be70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.207+0000 7f5f3cc00640 1 -- 172.21.15.90:0/2638436568 >> 172.21.15.90:0/2638436568 conn(0x7f5f380d3bc0 msgr2=0x7f5f380d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:23.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.208+0000 7f5f3cc00640 1 -- 172.21.15.90:0/2638436568 shutdown_connections 2024-03-16T17:57:23.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.208+0000 7f5f3cc00640 1 -- 172.21.15.90:0/2638436568 wait complete. 2024-03-16T17:57:23.228 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new 61bf727a-c14b-4cfd-97e0-d4156d1cd362 3 2024-03-16T17:57:23.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.461+0000 7fdc93e00640 1 Processor -- start 2024-03-16T17:57:23.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.461+0000 7fdc93e00640 1 -- start start 2024-03-16T17:57:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.462+0000 7fdc93e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fdc940d05a0 0x7fdc940d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.463+0000 7fdc93e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc940d0f40 con 0x7fdc9415e050 2024-03-16T17:57:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.463+0000 7fdc93e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc940cf640 con 0x7fdc940b3e40 2024-03-16T17:57:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.463+0000 7fdc93e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc940cf920 con 0x7fdc940d05a0 2024-03-16T17:57:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.463+0000 7fdc8be00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fdc940d05a0 0x7fdc940d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.463+0000 7fdc93400640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fdc940b3e40 0x7fdc940b4280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:42146/0 (socket says 172.21.15.90:42146) 2024-03-16T17:57:23.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.463+0000 7fdc93400640 1 -- 172.21.15.90:0/3511921670 learned_addr learned my addr 172.21.15.90:0/3511921670 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-16T17:57:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.463+0000 7fdc8be00640 1 -- 172.21.15.90:0/3511921670 >> v1:172.21.15.90:6789/0 conn(0x7fdc940b3e40 legacy=0x7fdc940b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.463+0000 7fdc8be00640 1 -- 172.21.15.90:0/3511921670 >> v1:172.21.15.82:6789/0 conn(0x7fdc9415e050 legacy=0x7fdc940d4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.463+0000 7fdc8be00640 1 -- 172.21.15.90:0/3511921670 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc940cfc00 con 0x7fdc940d05a0 2024-03-16T17:57:23.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.465+0000 7fdc8be00640 1 --2- 172.21.15.90:0/3511921670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc940d05a0 0x7fdc940d0a00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0097a0 tx=0x7fdc7c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4c0a15a72434df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:23.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.465+0000 7fdc92000640 1 -- 172.21.15.90:0/3511921670 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c031830 con 0x7fdc940d05a0 2024-03-16T17:57:23.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.465+0000 7fdc92000640 1 -- 172.21.15.90:0/3511921670 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c031990 con 0x7fdc940d05a0 2024-03-16T17:57:23.469 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.465+0000 7fdc92000640 1 -- 172.21.15.90:0/3511921670 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c031d20 con 0x7fdc940d05a0 2024-03-16T17:57:23.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.466+0000 7fdc93e00640 1 -- 172.21.15.90:0/3511921670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc940d05a0 msgr2=0x7fdc940d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.466+0000 7fdc93e00640 1 --2- 172.21.15.90:0/3511921670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc940d05a0 0x7fdc940d0a00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c0097a0 tx=0x7fdc7c02b8d0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.466+0000 7fdc93e00640 1 -- 172.21.15.90:0/3511921670 shutdown_connections 2024-03-16T17:57:23.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.466+0000 7fdc93e00640 1 --2- 172.21.15.90:0/3511921670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc940d05a0 0x7fdc940d0a00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.466+0000 7fdc93e00640 1 -- 172.21.15.90:0/3511921670 >> 172.21.15.90:0/3511921670 conn(0x7fdc9409cf30 msgr2=0x7fdc940d3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:23.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.466+0000 7fdc93e00640 1 -- 172.21.15.90:0/3511921670 shutdown_connections 2024-03-16T17:57:23.470 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.466+0000 7fdc93e00640 1 -- 172.21.15.90:0/3511921670 wait complete. 2024-03-16T17:57:23.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.468+0000 7fdc93e00640 1 Processor -- start 2024-03-16T17:57:23.471 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.468+0000 7fdc93e00640 1 -- start start 2024-03-16T17:57:23.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc93e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc940b3e40 0x7fdc941570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc93e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc940d05a0 0x7fdc940c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc93e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc9415e050 0x7fdc940ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc93e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc940c8240 con 0x7fdc940d05a0 2024-03-16T17:57:23.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc93e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc940c8520 con 0x7fdc9415e050 2024-03-16T17:57:23.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc93e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc940c8800 con 0x7fdc940b3e40 2024-03-16T17:57:23.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc92a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc940b3e40 0x7fdc941570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc92a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc940b3e40 0x7fdc941570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:33904/0 (socket says 172.21.15.90:33904) 2024-03-16T17:57:23.473 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc92a00640 1 -- 172.21.15.90:0/1492140151 learned_addr learned my addr 172.21.15.90:0/1492140151 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:23.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc93400640 1 --2- 172.21.15.90:0/1492140151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc9415e050 0x7fdc940ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.469+0000 7fdc8be00640 1 --2- 172.21.15.90:0/1492140151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc940d05a0 0x7fdc940c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc92a00640 1 -- 172.21.15.90:0/1492140151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc9415e050 msgr2=0x7fdc940ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc92a00640 1 --2- 172.21.15.90:0/1492140151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc9415e050 0x7fdc940ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc92a00640 1 -- 172.21.15.90:0/1492140151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc940d05a0 msgr2=0x7fdc940c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc92a00640 1 --2- 172.21.15.90:0/1492140151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc940d05a0 0x7fdc940c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc92a00640 1 -- 172.21.15.90:0/1492140151 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c009400 con 0x7fdc940b3e40 2024-03-16T17:57:23.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc8be00640 1 --2- 172.21.15.90:0/1492140151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc940d05a0 0x7fdc940c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:23.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc92a00640 1 --2- 172.21.15.90:0/1492140151 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc940b3e40 0x7fdc941570a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c400 tx=0x7fdc8400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:23.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc8aa00640 1 -- 172.21.15.90:0/1492140151 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc840052d0 con 0x7fdc940b3e40 2024-03-16T17:57:23.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc8aa00640 1 -- 172.21.15.90:0/1492140151 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc84005430 con 0x7fdc940b3e40 2024-03-16T17:57:23.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc8aa00640 1 -- 172.21.15.90:0/1492140151 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84003830 con 0x7fdc940b3e40 2024-03-16T17:57:23.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc93e00640 1 -- 172.21.15.90:0/1492140151 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc940c8ae0 con 0x7fdc940b3e40 2024-03-16T17:57:23.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc93e00640 1 -- 172.21.15.90:0/1492140151 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc940c92e0 con 0x7fdc940b3e40 2024-03-16T17:57:23.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.470+0000 7fdc8aa00640 1 -- 172.21.15.90:0/1492140151 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdc84003990 con 0x7fdc940b3e40 2024-03-16T17:57:23.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.471+0000 7fdc8aa00640 1 -- 172.21.15.90:0/1492140151 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fdc84003c40 con 0x7fdc940b3e40 2024-03-16T17:57:23.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.471+0000 7fdc93e00640 1 -- 172.21.15.90:0/1492140151 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc940c9000 con 0x7fdc940b3e40 2024-03-16T17:57:23.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.474+0000 7fdc8aa00640 1 -- 172.21.15.90:0/1492140151 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdc940c9000 con 0x7fdc940b3e40 2024-03-16T17:57:23.488 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:23.485+0000 7f3dda63f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T17:57:23.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.552+0000 7fdc93e00640 1 -- 172.21.15.90:0/1492140151 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd new", "uuid": "61bf727a-c14b-4cfd-97e0-d4156d1cd362", "id": 3} v 0) v1 -- 0x7fdc9404f250 con 0x7fdc940b3e40 2024-03-16T17:57:23.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.561+0000 7fdc8aa00640 1 -- 172.21.15.90:0/1492140151 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "61bf727a-c14b-4cfd-97e0-d4156d1cd362", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdc84021060 con 0x7fdc940b3e40 2024-03-16T17:57:23.565 INFO:teuthology.orchestra.run.smithi090.stdout:3 2024-03-16T17:57:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.562+0000 7fdc93e00640 1 -- 172.21.15.90:0/1492140151 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc940b3e40 msgr2=0x7fdc941570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.562+0000 7fdc93e00640 1 --2- 172.21.15.90:0/1492140151 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc940b3e40 0x7fdc941570a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c400 tx=0x7fdc8400c8c0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.562+0000 7fdc93e00640 1 -- 172.21.15.90:0/1492140151 shutdown_connections 2024-03-16T17:57:23.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.562+0000 7fdc93e00640 1 --2- 172.21.15.90:0/1492140151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc9415e050 0x7fdc940ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.562+0000 7fdc93e00640 1 --2- 172.21.15.90:0/1492140151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc940d05a0 0x7fdc940c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.562+0000 7fdc93e00640 1 --2- 172.21.15.90:0/1492140151 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc940b3e40 0x7fdc941570a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.562+0000 7fdc93e00640 1 -- 172.21.15.90:0/1492140151 >> 172.21.15.90:0/1492140151 conn(0x7fdc9409cf30 msgr2=0x7fdc940d17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:23.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.562+0000 7fdc93e00640 1 -- 172.21.15.90:0/1492140151 shutdown_connections 2024-03-16T17:57:23.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.562+0000 7fdc93e00640 1 -- 172.21.15.90:0/1492140151 wait complete. 2024-03-16T17:57:23.582 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new fcb62abd-4b1f-4f86-a989-88111ec0e595 4 2024-03-16T17:57:23.589 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:23.585+0000 7f798dcda040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-16T17:57:23.647 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:23.643+0000 7f3dda63f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T17:57:23.756 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:23.752+0000 7f798dcda040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T17:57:23.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.772+0000 7fe9ffa00640 1 Processor -- start 2024-03-16T17:57:23.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.772+0000 7fe9ffa00640 1 -- start start 2024-03-16T17:57:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.774+0000 7fe9ffa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9f80d2ab0 0x7fe9f815e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.774+0000 7fe9ffa00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f80d48e0 0x7fe9f80ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.774+0000 7fe9ffa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f80b33c0 con 0x7fe9f815e040 2024-03-16T17:57:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.774+0000 7fe9ffa00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f80d1cd0 con 0x7fe9f80d2ab0 2024-03-16T17:57:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.774+0000 7fe9ffa00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80d1fb0 con 0x7fe9f80d48e0 2024-03-16T17:57:23.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.775+0000 7fe9fda00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f80d48e0 0x7fe9f80ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.775+0000 7fe9fda00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f80d48e0 0x7fe9f80ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:33908/0 (socket says 172.21.15.90:33908) 2024-03-16T17:57:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.775+0000 7fe9fda00640 1 -- 172.21.15.90:0/1760572197 learned_addr learned my addr 172.21.15.90:0/1760572197 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.775+0000 7fe9f7e00640 1 --2- 172.21.15.90:0/1760572197 >> v2:172.21.15.82:3300/0 conn(0x7fe9f80d2ab0 0x7fe9f815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.775+0000 7fe9fda00640 1 -- 172.21.15.90:0/1760572197 >> v2:172.21.15.82:3300/0 conn(0x7fe9f80d2ab0 msgr2=0x7fe9f815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.775+0000 7fe9fda00640 1 --2- 172.21.15.90:0/1760572197 >> v2:172.21.15.82:3300/0 conn(0x7fe9f80d2ab0 0x7fe9f815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.775+0000 7fe9fda00640 1 -- 172.21.15.90:0/1760572197 >> v1:172.21.15.82:6789/0 conn(0x7fe9f815e040 legacy=0x7fe9f80d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.775+0000 7fe9fda00640 1 -- 172.21.15.90:0/1760572197 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80b3610 con 0x7fe9f80d48e0 2024-03-16T17:57:23.781 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:23.776+0000 7f3dda63f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T17:57:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.776+0000 7fe9fda00640 1 --2- 172.21.15.90:0/1760572197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f80d48e0 0x7fe9f80ce9d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009e50 tx=0x7fe9f002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f700eb155e93c437 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.776+0000 7fe9f7400640 1 -- 172.21.15.90:0/1760572197 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f002ca70 con 0x7fe9f80d48e0 2024-03-16T17:57:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.776+0000 7fe9f7400640 1 -- 172.21.15.90:0/1760572197 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f002cbd0 con 0x7fe9f80d48e0 2024-03-16T17:57:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.776+0000 7fe9f7400640 1 -- 172.21.15.90:0/1760572197 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00343f0 con 0x7fe9f80d48e0 2024-03-16T17:57:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.777+0000 7fe9ffa00640 1 -- 172.21.15.90:0/1760572197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f80d48e0 msgr2=0x7fe9f80ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.777+0000 7fe9ffa00640 1 --2- 172.21.15.90:0/1760572197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f80d48e0 0x7fe9f80ce9d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009e50 tx=0x7fe9f002bed0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.777+0000 7fe9ffa00640 1 -- 172.21.15.90:0/1760572197 shutdown_connections 2024-03-16T17:57:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.777+0000 7fe9ffa00640 1 --2- 172.21.15.90:0/1760572197 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f80d48e0 0x7fe9f80ce9d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.777+0000 7fe9ffa00640 1 --2- 172.21.15.90:0/1760572197 >> v2:172.21.15.82:3300/0 conn(0x7fe9f80d2ab0 0x7fe9f815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.777+0000 7fe9ffa00640 1 -- 172.21.15.90:0/1760572197 >> 172.21.15.90:0/1760572197 conn(0x7fe9f80d3dc0 msgr2=0x7fe9f80d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.777+0000 7fe9ffa00640 1 -- 172.21.15.90:0/1760572197 shutdown_connections 2024-03-16T17:57:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.777+0000 7fe9ffa00640 1 -- 172.21.15.90:0/1760572197 wait complete. 2024-03-16T17:57:23.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.780+0000 7fe9ffa00640 1 Processor -- start 2024-03-16T17:57:23.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.780+0000 7fe9ffa00640 1 -- start start 2024-03-16T17:57:23.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.781+0000 7fe9ffa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9f80d2ab0 0x7fe9f815c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.781+0000 7fe9ffa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f80d48e0 0x7fe9f80cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.781+0000 7fe9ffa00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f815e040 0x7fe9f80a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:23.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.781+0000 7fe9ffa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f809f230 con 0x7fe9f80d48e0 2024-03-16T17:57:23.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.781+0000 7fe9ffa00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80cc9b0 con 0x7fe9f80d2ab0 2024-03-16T17:57:23.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.781+0000 7fe9ffa00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80cb9d0 con 0x7fe9f815e040 2024-03-16T17:57:23.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9fda00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f815e040 0x7fe9f80a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9fd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9f80d2ab0 0x7fe9f815c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9fda00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f815e040 0x7fe9f80a1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:33912/0 (socket says 172.21.15.90:33912) 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9fda00640 1 -- 172.21.15.90:0/3496469698 learned_addr learned my addr 172.21.15.90:0/3496469698 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9f7e00640 1 --2- 172.21.15.90:0/3496469698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f80d48e0 0x7fe9f80cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9fda00640 1 -- 172.21.15.90:0/3496469698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9f80d2ab0 msgr2=0x7fe9f815c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9fda00640 1 --2- 172.21.15.90:0/3496469698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9f80d2ab0 0x7fe9f815c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9fda00640 1 -- 172.21.15.90:0/3496469698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f80d48e0 msgr2=0x7fe9f80cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9fda00640 1 --2- 172.21.15.90:0/3496469698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f80d48e0 0x7fe9f80cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9fda00640 1 -- 172.21.15.90:0/3496469698 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0009b30 con 0x7fe9f815e040 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9f7e00640 1 --2- 172.21.15.90:0/3496469698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f80d48e0 0x7fe9f80cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.782+0000 7fe9fda00640 1 --2- 172.21.15.90:0/3496469698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f815e040 0x7fe9f80a1620 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009e50 tx=0x7fe9f0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.783+0000 7fe9f4c00640 1 -- 172.21.15.90:0/3496469698 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0009510 con 0x7fe9f815e040 2024-03-16T17:57:23.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.783+0000 7fe9f4c00640 1 -- 172.21.15.90:0/3496469698 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f003e450 con 0x7fe9f815e040 2024-03-16T17:57:23.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.783+0000 7fe9f4c00640 1 -- 172.21.15.90:0/3496469698 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f003e800 con 0x7fe9f815e040 2024-03-16T17:57:23.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.783+0000 7fe9ffa00640 1 -- 172.21.15.90:0/3496469698 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f80cd610 con 0x7fe9f815e040 2024-03-16T17:57:23.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.783+0000 7fe9ffa00640 1 -- 172.21.15.90:0/3496469698 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f809ef50 con 0x7fe9f815e040 2024-03-16T17:57:23.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.783+0000 7fe9f4c00640 1 -- 172.21.15.90:0/3496469698 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe9f003e960 con 0x7fe9f815e040 2024-03-16T17:57:23.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.783+0000 7fe9f4c00640 1 -- 172.21.15.90:0/3496469698 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fe9f003ec10 con 0x7fe9f815e040 2024-03-16T17:57:23.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.784+0000 7fe9ffa00640 1 -- 172.21.15.90:0/3496469698 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f815ca20 con 0x7fe9f815e040 2024-03-16T17:57:23.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.788+0000 7fe9f4c00640 1 -- 172.21.15.90:0/3496469698 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe9f0066dd0 con 0x7fe9f815e040 2024-03-16T17:57:23.834 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:23.831+0000 7f3dda63f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T17:57:23.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.868+0000 7fe9ffa00640 1 -- 172.21.15.90:0/3496469698 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fcb62abd-4b1f-4f86-a989-88111ec0e595", "id": 4} v 0) v1 -- 0x7fe9f8067f30 con 0x7fe9f815e040 2024-03-16T17:57:23.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.875+0000 7fe9f4c00640 1 -- 172.21.15.90:0/3496469698 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fcb62abd-4b1f-4f86-a989-88111ec0e595", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe9f0055030 con 0x7fe9f815e040 2024-03-16T17:57:23.879 INFO:teuthology.orchestra.run.smithi090.stdout:4 2024-03-16T17:57:23.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.876+0000 7fe9ffa00640 1 -- 172.21.15.90:0/3496469698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f815e040 msgr2=0x7fe9f80a1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:23.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.876+0000 7fe9ffa00640 1 --2- 172.21.15.90:0/3496469698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f815e040 0x7fe9f80a1620 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009e50 tx=0x7fe9f0004a40 comp rx=0 tx=0).stop 2024-03-16T17:57:23.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.876+0000 7fe9ffa00640 1 -- 172.21.15.90:0/3496469698 shutdown_connections 2024-03-16T17:57:23.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.876+0000 7fe9ffa00640 1 --2- 172.21.15.90:0/3496469698 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe9f815e040 0x7fe9f80a1620 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.876+0000 7fe9ffa00640 1 --2- 172.21.15.90:0/3496469698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f80d48e0 0x7fe9f80cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.876+0000 7fe9ffa00640 1 --2- 172.21.15.90:0/3496469698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9f80d2ab0 0x7fe9f815c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:23.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.877+0000 7fe9ffa00640 1 -- 172.21.15.90:0/3496469698 >> 172.21.15.90:0/3496469698 conn(0x7fe9f80d3dc0 msgr2=0x7fe9f80cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:23.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.877+0000 7fe9ffa00640 1 -- 172.21.15.90:0/3496469698 shutdown_connections 2024-03-16T17:57:23.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:23.877+0000 7fe9ffa00640 1 -- 172.21.15.90:0/3496469698 wait complete. 2024-03-16T17:57:23.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new e38bdb1d-ad07-4368-84b8-a6e6afa0ae93 5 2024-03-16T17:57:23.915 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:23.911+0000 7f798dcda040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T17:57:23.962 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:23.959+0000 7f3dda63f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T17:57:23.972 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:23.968+0000 7f798dcda040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T17:57:24.072 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:24.069+0000 7f3dda63f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T17:57:24.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.078+0000 7f24a9400640 1 Processor -- start 2024-03-16T17:57:24.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.079+0000 7f24a9400640 1 -- start start 2024-03-16T17:57:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.080+0000 7f24a9400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f24a415e000 0x7f24a40d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.080+0000 7f24a9400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a40b3910 con 0x7f24a40d0ca0 2024-03-16T17:57:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.080+0000 7f24a9400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a40d2580 con 0x7f24a40cf9e0 2024-03-16T17:57:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.080+0000 7f24a9400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a415e3e0 con 0x7f24a415e000 2024-03-16T17:57:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.080+0000 7f24a3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f24a415e000 0x7f24a40d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.080+0000 7f24a3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f24a415e000 0x7f24a40d46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56512/0 (socket says 172.21.15.90:56512) 2024-03-16T17:57:24.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.080+0000 7f24a3e00640 1 -- 172.21.15.90:0/4169042189 learned_addr learned my addr 172.21.15.90:0/4169042189 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:24.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.081+0000 7f24a3e00640 1 -- 172.21.15.90:0/4169042189 >> v1:172.21.15.90:6789/0 conn(0x7f24a40cf9e0 legacy=0x7f24a40d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.081+0000 7f24a3e00640 1 -- 172.21.15.90:0/4169042189 >> v1:172.21.15.82:6789/0 conn(0x7f24a40d0ca0 legacy=0x7f24a40b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.081+0000 7f24a3e00640 1 -- 172.21.15.90:0/4169042189 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a40d2860 con 0x7f24a415e000 2024-03-16T17:57:24.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.081+0000 7f24a3400640 1 -- 172.21.15.90:0/4169042189 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272061990 0 0) 0x7f24a40d2580 con 0x7f24a40cf9e0 2024-03-16T17:57:24.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.082+0000 7f24a3e00640 1 --2- 172.21.15.90:0/4169042189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24a415e000 0x7f24a40d46e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2494009870 tx=0x7f249402ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd42d1402667f198 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:24.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.082+0000 7f24a3400640 1 -- 172.21.15.90:0/4169042189 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249402f840 con 0x7f24a415e000 2024-03-16T17:57:24.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.082+0000 7f24a3400640 1 -- 172.21.15.90:0/4169042189 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249402f9a0 con 0x7f24a415e000 2024-03-16T17:57:24.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.082+0000 7f24a3400640 1 -- 172.21.15.90:0/4169042189 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249402fd30 con 0x7f24a415e000 2024-03-16T17:57:24.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.083+0000 7f24a9400640 1 -- 172.21.15.90:0/4169042189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24a415e000 msgr2=0x7f24a40d46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.083+0000 7f24a9400640 1 --2- 172.21.15.90:0/4169042189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24a415e000 0x7f24a40d46e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2494009870 tx=0x7f249402ba20 comp rx=0 tx=0).stop 2024-03-16T17:57:24.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.083+0000 7f24a9400640 1 -- 172.21.15.90:0/4169042189 shutdown_connections 2024-03-16T17:57:24.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.083+0000 7f24a9400640 1 --2- 172.21.15.90:0/4169042189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24a415e000 0x7f24a40d46e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.083+0000 7f24a9400640 1 -- 172.21.15.90:0/4169042189 >> 172.21.15.90:0/4169042189 conn(0x7f24a40d3bc0 msgr2=0x7f24a40d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:24.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.083+0000 7f24a9400640 1 -- 172.21.15.90:0/4169042189 shutdown_connections 2024-03-16T17:57:24.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.083+0000 7f24a9400640 1 -- 172.21.15.90:0/4169042189 wait complete. 2024-03-16T17:57:24.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.085+0000 7f24a9400640 1 Processor -- start 2024-03-16T17:57:24.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.085+0000 7f24a9400640 1 -- start start 2024-03-16T17:57:24.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a9400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f24a40cf9e0 0x7f24a41565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a9400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24a40d0ca0 0x7f24a4152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a9400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a415e000 0x7f24a40b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a9400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a40cb050 con 0x7f24a415e000 2024-03-16T17:57:24.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a9400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a40cb330 con 0x7f24a40d0ca0 2024-03-16T17:57:24.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a9400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a40cb610 con 0x7f24a40cf9e0 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a3e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f24a40cf9e0 0x7f24a41565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a3e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f24a40cf9e0 0x7f24a41565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.90:33924/0 (socket says 172.21.15.90:33924) 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a3e00640 1 -- 172.21.15.90:0/3643859063 learned_addr learned my addr 172.21.15.90:0/3643859063 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f249be00640 1 --2- 172.21.15.90:0/3643859063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24a40d0ca0 0x7f24a4152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a8a00640 1 --2- 172.21.15.90:0/3643859063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a415e000 0x7f24a40b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a3e00640 1 -- 172.21.15.90:0/3643859063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24a40d0ca0 msgr2=0x7f24a4152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a3e00640 1 --2- 172.21.15.90:0/3643859063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24a40d0ca0 0x7f24a4152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a3e00640 1 -- 172.21.15.90:0/3643859063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a415e000 msgr2=0x7f24a40b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a3e00640 1 --2- 172.21.15.90:0/3643859063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a415e000 0x7f24a40b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f24a3e00640 1 -- 172.21.15.90:0/3643859063 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24940094d0 con 0x7f24a40cf9e0 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.086+0000 7f249be00640 1 --2- 172.21.15.90:0/3643859063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24a40d0ca0 0x7f24a4152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.087+0000 7f24a3e00640 1 --2- 172.21.15.90:0/3643859063 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f24a40cf9e0 0x7f24a41565a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2494009870 tx=0x7f2494004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.087+0000 7f24a0c00640 1 -- 172.21.15.90:0/3643859063 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249402f7f0 con 0x7f24a40cf9e0 2024-03-16T17:57:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.087+0000 7f24a0c00640 1 -- 172.21.15.90:0/3643859063 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2494041400 con 0x7f24a40cf9e0 2024-03-16T17:57:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.087+0000 7f24a0c00640 1 -- 172.21.15.90:0/3643859063 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24940417b0 con 0x7f24a40cf9e0 2024-03-16T17:57:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.087+0000 7f24a9400640 1 -- 172.21.15.90:0/3643859063 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a40cb8f0 con 0x7f24a40cf9e0 2024-03-16T17:57:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.087+0000 7f24a9400640 1 -- 172.21.15.90:0/3643859063 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a4156ae0 con 0x7f24a40cf9e0 2024-03-16T17:57:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.087+0000 7f24a0c00640 1 -- 172.21.15.90:0/3643859063 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2494041910 con 0x7f24a40cf9e0 2024-03-16T17:57:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.087+0000 7f24a0c00640 1 -- 172.21.15.90:0/3643859063 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f249403c070 con 0x7f24a40cf9e0 2024-03-16T17:57:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.088+0000 7f24a9400640 1 -- 172.21.15.90:0/3643859063 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a404f250 con 0x7f24a40cf9e0 2024-03-16T17:57:24.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.092+0000 7f24a0c00640 1 -- 172.21.15.90:0/3643859063 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2494041bc0 con 0x7f24a40cf9e0 2024-03-16T17:57:24.107 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:24.103+0000 7f798dcda040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T17:57:24.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.172+0000 7f24a9400640 1 -- 172.21.15.90:0/3643859063 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e38bdb1d-ad07-4368-84b8-a6e6afa0ae93", "id": 5} v 0) v1 -- 0x7f24a40b2ec0 con 0x7f24a40cf9e0 2024-03-16T17:57:24.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.178+0000 7f24a0c00640 1 -- 172.21.15.90:0/3643859063 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e38bdb1d-ad07-4368-84b8-a6e6afa0ae93", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f249404b460 con 0x7f24a40cf9e0 2024-03-16T17:57:24.182 INFO:teuthology.orchestra.run.smithi090.stdout:5 2024-03-16T17:57:24.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.179+0000 7f24a9400640 1 -- 172.21.15.90:0/3643859063 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f24a40cf9e0 msgr2=0x7f24a41565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.180+0000 7f24a9400640 1 --2- 172.21.15.90:0/3643859063 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f24a40cf9e0 0x7f24a41565a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2494009870 tx=0x7f2494004730 comp rx=0 tx=0).stop 2024-03-16T17:57:24.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.180+0000 7f24a9400640 1 -- 172.21.15.90:0/3643859063 shutdown_connections 2024-03-16T17:57:24.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.180+0000 7f24a9400640 1 --2- 172.21.15.90:0/3643859063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24a415e000 0x7f24a40b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.180+0000 7f24a9400640 1 --2- 172.21.15.90:0/3643859063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f24a40d0ca0 0x7f24a4152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.180+0000 7f24a9400640 1 --2- 172.21.15.90:0/3643859063 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f24a40cf9e0 0x7f24a41565a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.180+0000 7f24a9400640 1 -- 172.21.15.90:0/3643859063 >> 172.21.15.90:0/3643859063 conn(0x7f24a40d3bc0 msgr2=0x7f24a40d4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:24.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.180+0000 7f24a9400640 1 -- 172.21.15.90:0/3643859063 shutdown_connections 2024-03-16T17:57:24.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.180+0000 7f24a9400640 1 -- 172.21.15.90:0/3643859063 wait complete. 2024-03-16T17:57:24.200 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new e08850eb-1b52-4cb7-bbcb-265031c59075 6 2024-03-16T17:57:24.226 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:24.222+0000 7f798dcda040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T17:57:24.251 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:24.247+0000 7f3dda63f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T17:57:24.301 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:24.297+0000 7f3dda63f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T17:57:24.412 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:24.407+0000 7f798dcda040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T17:57:24.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.414+0000 7f020fe00640 1 Processor -- start 2024-03-16T17:57:24.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.414+0000 7f020fe00640 1 -- start start 2024-03-16T17:57:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.415+0000 7f020fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f02101454d0 0x7f0210160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.415+0000 7f020fe00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f021006e010 0x7f02100cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.415+0000 7f020fe00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02101458b0 con 0x7f021006e650 2024-03-16T17:57:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.415+0000 7f020fe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02100cf3b0 con 0x7f02101454d0 2024-03-16T17:57:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.415+0000 7f020fe00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02100cf690 con 0x7f021006e010 2024-03-16T17:57:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.416+0000 7f020f400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f021006e010 0x7f02100cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.416+0000 7f020e000640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f021006e650 0x7f021006d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:42172/0 (socket says 172.21.15.90:42172) 2024-03-16T17:57:24.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.416+0000 7f020e000640 1 -- 172.21.15.90:0/743206529 learned_addr learned my addr 172.21.15.90:0/743206529 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-16T17:57:24.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.416+0000 7f020ea00640 1 --2- 172.21.15.90:0/743206529 >> v2:172.21.15.82:3300/0 conn(0x7f02101454d0 0x7f0210160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.416+0000 7f020f400640 1 -- 172.21.15.90:0/743206529 >> v2:172.21.15.82:3300/0 conn(0x7f02101454d0 msgr2=0x7f0210160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:24.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.416+0000 7f020f400640 1 --2- 172.21.15.90:0/743206529 >> v2:172.21.15.82:3300/0 conn(0x7f02101454d0 0x7f0210160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.416+0000 7f020f400640 1 -- 172.21.15.90:0/743206529 >> v1:172.21.15.90:6789/0 conn(0x7f021006e650 legacy=0x7f021006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.416+0000 7f020f400640 1 -- 172.21.15.90:0/743206529 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021015f7a0 con 0x7f021006e010 2024-03-16T17:57:24.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.417+0000 7f020f400640 1 --2- 172.21.15.90:0/743206529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 0x7f02100cd6f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f02100b1640 tx=0x7f020802bec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdcdb3e1a0857536 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:24.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.417+0000 7f020d600640 1 -- 172.21.15.90:0/743206529 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020802cc70 con 0x7f021006e010 2024-03-16T17:57:24.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.418+0000 7f020d600640 1 -- 172.21.15.90:0/743206529 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020802cdd0 con 0x7f021006e010 2024-03-16T17:57:24.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.418+0000 7f020d600640 1 -- 172.21.15.90:0/743206529 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208034660 con 0x7f021006e010 2024-03-16T17:57:24.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.418+0000 7f020fe00640 1 -- 172.21.15.90:0/743206529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 msgr2=0x7f02100cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.418+0000 7f020fe00640 1 --2- 172.21.15.90:0/743206529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 0x7f02100cd6f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f02100b1640 tx=0x7f020802bec0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.418+0000 7f020fe00640 1 -- 172.21.15.90:0/743206529 shutdown_connections 2024-03-16T17:57:24.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.418+0000 7f020fe00640 1 --2- 172.21.15.90:0/743206529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 0x7f02100cd6f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.418+0000 7f020fe00640 1 --2- 172.21.15.90:0/743206529 >> v2:172.21.15.82:3300/0 conn(0x7f02101454d0 0x7f0210160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.418+0000 7f020fe00640 1 -- 172.21.15.90:0/743206529 >> 172.21.15.90:0/743206529 conn(0x7f021006ae70 msgr2=0x7f021006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:24.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.418+0000 7f020fe00640 1 -- 172.21.15.90:0/743206529 shutdown_connections 2024-03-16T17:57:24.423 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.418+0000 7f020fe00640 1 -- 172.21.15.90:0/743206529 wait complete. 2024-03-16T17:57:24.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.420+0000 7f020fe00640 1 Processor -- start 2024-03-16T17:57:24.424 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.420+0000 7f020fe00640 1 -- start start 2024-03-16T17:57:24.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020fe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 0x7f021012eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020fe00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f021006e650 0x7f021015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101454d0 0x7f02100a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020fe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021015ba50 con 0x7f02101454d0 2024-03-16T17:57:24.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020fe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021012fd00 con 0x7f021006e010 2024-03-16T17:57:24.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020fe00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021013d5f0 con 0x7f021006e650 2024-03-16T17:57:24.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 0x7f021012eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 0x7f021012eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56524/0 (socket says 172.21.15.90:56524) 2024-03-16T17:57:24.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020ea00640 1 -- 172.21.15.90:0/2628568951 learned_addr learned my addr 172.21.15.90:0/2628568951 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:24.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020e000640 1 --2- 172.21.15.90:0/2628568951 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f021006e650 0x7f021015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.426 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.421+0000 7f020f400640 1 --2- 172.21.15.90:0/2628568951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101454d0 0x7f02100a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f020ea00640 1 -- 172.21.15.90:0/2628568951 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f021006e650 msgr2=0x7f021015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f020ea00640 1 --2- 172.21.15.90:0/2628568951 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f021006e650 0x7f021015b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f020ea00640 1 -- 172.21.15.90:0/2628568951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101454d0 msgr2=0x7f02100a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f020ea00640 1 --2- 172.21.15.90:0/2628568951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101454d0 0x7f02100a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f020ea00640 1 -- 172.21.15.90:0/2628568951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0208009a70 con 0x7f021006e010 2024-03-16T17:57:24.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f020f400640 1 --2- 172.21.15.90:0/2628568951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101454d0 0x7f02100a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:24.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f020ea00640 1 --2- 172.21.15.90:0/2628568951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 0x7f021012eda0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f020000d5f0 tx=0x7f020000dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:24.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f0206a00640 1 -- 172.21.15.90:0/2628568951 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02000125f0 con 0x7f021006e010 2024-03-16T17:57:24.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f0206a00640 1 -- 172.21.15.90:0/2628568951 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0200012750 con 0x7f021006e010 2024-03-16T17:57:24.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f0206a00640 1 -- 172.21.15.90:0/2628568951 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200012b00 con 0x7f021006e010 2024-03-16T17:57:24.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f020fe00640 1 -- 172.21.15.90:0/2628568951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021012f460 con 0x7f021006e010 2024-03-16T17:57:24.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.422+0000 7f020fe00640 1 -- 172.21.15.90:0/2628568951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021012fa20 con 0x7f021006e010 2024-03-16T17:57:24.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.423+0000 7f0206a00640 1 -- 172.21.15.90:0/2628568951 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0200012c60 con 0x7f021006e010 2024-03-16T17:57:24.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.423+0000 7f0206a00640 1 -- 172.21.15.90:0/2628568951 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f020001ea50 con 0x7f021006e010 2024-03-16T17:57:24.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.424+0000 7f020fe00640 1 -- 172.21.15.90:0/2628568951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021004f250 con 0x7f021006e010 2024-03-16T17:57:24.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.426+0000 7f0206a00640 1 -- 172.21.15.90:0/2628568951 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f020001e460 con 0x7f021006e010 2024-03-16T17:57:24.475 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:24.471+0000 7f3dda63f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T17:57:24.482 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:24.479+0000 7f798dcda040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T17:57:24.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.521+0000 7f020fe00640 1 -- 172.21.15.90:0/2628568951 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e08850eb-1b52-4cb7-bbcb-265031c59075", "id": 6} v 0) v1 -- 0x7f021006e430 con 0x7f021006e010 2024-03-16T17:57:24.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.529+0000 7f0206a00640 1 -- 172.21.15.90:0/2628568951 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e08850eb-1b52-4cb7-bbcb-265031c59075", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f020001e600 con 0x7f021006e010 2024-03-16T17:57:24.533 INFO:teuthology.orchestra.run.smithi090.stdout:6 2024-03-16T17:57:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.530+0000 7f020fe00640 1 -- 172.21.15.90:0/2628568951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 msgr2=0x7f021012eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.530+0000 7f020fe00640 1 --2- 172.21.15.90:0/2628568951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 0x7f021012eda0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f020000d5f0 tx=0x7f020000dab0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.530+0000 7f020fe00640 1 -- 172.21.15.90:0/2628568951 shutdown_connections 2024-03-16T17:57:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.530+0000 7f020fe00640 1 --2- 172.21.15.90:0/2628568951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02101454d0 0x7f02100a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.530+0000 7f020fe00640 1 --2- 172.21.15.90:0/2628568951 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f021006e650 0x7f021015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.530+0000 7f020fe00640 1 --2- 172.21.15.90:0/2628568951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f021006e010 0x7f021012eda0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.530+0000 7f020fe00640 1 -- 172.21.15.90:0/2628568951 >> 172.21.15.90:0/2628568951 conn(0x7f021006ae70 msgr2=0x7f021015bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.531+0000 7f020fe00640 1 -- 172.21.15.90:0/2628568951 shutdown_connections 2024-03-16T17:57:24.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.531+0000 7f020fe00640 1 -- 172.21.15.90:0/2628568951 wait complete. 2024-03-16T17:57:24.551 DEBUG:teuthology.orchestra.run.smithi090:> sudo ceph --cluster ceph osd new c948154a-8b9b-4a9a-967a-615651402176 7 2024-03-16T17:57:24.672 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:24.668+0000 7f798dcda040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T17:57:24.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.732+0000 7f9a18e00640 1 Processor -- start 2024-03-16T17:57:24.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.732+0000 7f9a18e00640 1 -- start start 2024-03-16T17:57:24.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.733+0000 7f9a18e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9a1415fcc0 0x7f9a1406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.733+0000 7f9a18e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a140b41c0 0x7f9a141600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.733+0000 7f9a18e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a141605e0 con 0x7f9a140cd8b0 2024-03-16T17:57:24.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.733+0000 7f9a18e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a140ccaf0 con 0x7f9a140b41c0 2024-03-16T17:57:24.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.733+0000 7f9a18e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a140ccdd0 con 0x7f9a1415fcc0 2024-03-16T17:57:24.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.734+0000 7f9a11e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9a1415fcc0 0x7f9a1406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.734+0000 7f9a12800640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f9a140cd8b0 0x7f9a140ce230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.90:42186/0 (socket says 172.21.15.90:42186) 2024-03-16T17:57:24.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.734+0000 7f9a12800640 1 -- 172.21.15.90:0/4138946953 learned_addr learned my addr 172.21.15.90:0/4138946953 (peer_addr_for_me v1:172.21.15.90:0/0) 2024-03-16T17:57:24.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.734+0000 7f9a0be00640 1 --2- 172.21.15.90:0/4138946953 >> v2:172.21.15.82:3300/0 conn(0x7f9a140b41c0 0x7f9a141600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.734+0000 7f9a11e00640 1 -- 172.21.15.90:0/4138946953 >> v2:172.21.15.82:3300/0 conn(0x7f9a140b41c0 msgr2=0x7f9a141600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:24.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.734+0000 7f9a11e00640 1 --2- 172.21.15.90:0/4138946953 >> v2:172.21.15.82:3300/0 conn(0x7f9a140b41c0 0x7f9a141600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.734+0000 7f9a11e00640 1 -- 172.21.15.90:0/4138946953 >> v1:172.21.15.90:6789/0 conn(0x7f9a140cd8b0 legacy=0x7f9a140ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.734+0000 7f9a11e00640 1 -- 172.21.15.90:0/4138946953 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140cd0b0 con 0x7f9a1415fcc0 2024-03-16T17:57:24.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.735+0000 7f9a11e00640 1 --2- 172.21.15.90:0/4138946953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a1415fcc0 0x7f9a1406ed50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9a14072f70 tx=0x7f9a0402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36c007f09793187a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:24.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.735+0000 7f9a11400640 1 -- 172.21.15.90:0/4138946953 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04031830 con 0x7f9a1415fcc0 2024-03-16T17:57:24.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.735+0000 7f9a11400640 1 -- 172.21.15.90:0/4138946953 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04031990 con 0x7f9a1415fcc0 2024-03-16T17:57:24.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.736+0000 7f9a11400640 1 -- 172.21.15.90:0/4138946953 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04031d20 con 0x7f9a1415fcc0 2024-03-16T17:57:24.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.736+0000 7f9a18e00640 1 -- 172.21.15.90:0/4138946953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a1415fcc0 msgr2=0x7f9a1406ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.736+0000 7f9a18e00640 1 --2- 172.21.15.90:0/4138946953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a1415fcc0 0x7f9a1406ed50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9a14072f70 tx=0x7f9a0402b8d0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.736+0000 7f9a18e00640 1 -- 172.21.15.90:0/4138946953 shutdown_connections 2024-03-16T17:57:24.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.736+0000 7f9a18e00640 1 --2- 172.21.15.90:0/4138946953 >> v2:172.21.15.82:3300/0 conn(0x7f9a140b41c0 0x7f9a141600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.736+0000 7f9a18e00640 1 --2- 172.21.15.90:0/4138946953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a1415fcc0 0x7f9a1406ed50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.736+0000 7f9a18e00640 1 -- 172.21.15.90:0/4138946953 >> 172.21.15.90:0/4138946953 conn(0x7f9a140b36a0 msgr2=0x7f9a140b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:24.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.736+0000 7f9a18e00640 1 -- 172.21.15.90:0/4138946953 shutdown_connections 2024-03-16T17:57:24.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.736+0000 7f9a18e00640 1 -- 172.21.15.90:0/4138946953 wait complete. 2024-03-16T17:57:24.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.738+0000 7f9a18e00640 1 Processor -- start 2024-03-16T17:57:24.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.738+0000 7f9a18e00640 1 -- start start 2024-03-16T17:57:24.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a18e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a140b41c0 0x7f9a141e6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a18e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a140cd8b0 0x7f9a140cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a18e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a1415fcc0 0x7f9a140ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a18e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141e7360 con 0x7f9a140b41c0 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a18e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140b5940 con 0x7f9a140cd8b0 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a18e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140cbf60 con 0x7f9a1415fcc0 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a140cd8b0 0x7f9a140cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a0be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a140cd8b0 0x7f9a140cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56546/0 (socket says 172.21.15.90:56546) 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a0be00640 1 -- 172.21.15.90:0/1889851955 learned_addr learned my addr 172.21.15.90:0/1889851955 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a12800640 1 --2- 172.21.15.90:0/1889851955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a1415fcc0 0x7f9a140ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.739+0000 7f9a11e00640 1 --2- 172.21.15.90:0/1889851955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a140b41c0 0x7f9a141e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.740+0000 7f9a0be00640 1 -- 172.21.15.90:0/1889851955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a1415fcc0 msgr2=0x7f9a140ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.740+0000 7f9a0be00640 1 --2- 172.21.15.90:0/1889851955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a1415fcc0 0x7f9a140ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.740+0000 7f9a0be00640 1 -- 172.21.15.90:0/1889851955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a140b41c0 msgr2=0x7f9a141e6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.740+0000 7f9a0be00640 1 --2- 172.21.15.90:0/1889851955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a140b41c0 0x7f9a141e6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.740+0000 7f9a0be00640 1 -- 172.21.15.90:0/1889851955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04009400 con 0x7f9a140cd8b0 2024-03-16T17:57:24.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.740+0000 7f9a0be00640 1 --2- 172.21.15.90:0/1889851955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a140cd8b0 0x7f9a140cba20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00d5f0 tx=0x7f99fc00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.740+0000 7f9a0a000640 1 -- 172.21.15.90:0/1889851955 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0125f0 con 0x7f9a140cd8b0 2024-03-16T17:57:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.740+0000 7f9a0a000640 1 -- 172.21.15.90:0/1889851955 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99fc012750 con 0x7f9a140cd8b0 2024-03-16T17:57:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.740+0000 7f9a0a000640 1 -- 172.21.15.90:0/1889851955 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99fc012b00 con 0x7f9a140cd8b0 2024-03-16T17:57:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.740+0000 7f9a18e00640 1 -- 172.21.15.90:0/1889851955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a141e6ac0 con 0x7f9a140cd8b0 2024-03-16T17:57:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.741+0000 7f9a18e00640 1 -- 172.21.15.90:0/1889851955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a141e7080 con 0x7f9a140cd8b0 2024-03-16T17:57:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.741+0000 7f9a0a000640 1 -- 172.21.15.90:0/1889851955 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f99fc012c60 con 0x7f9a140cd8b0 2024-03-16T17:57:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.741+0000 7f9a0a000640 1 -- 172.21.15.90:0/1889851955 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f99fc01eac0 con 0x7f9a140cd8b0 2024-03-16T17:57:24.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.742+0000 7f9a18e00640 1 -- 172.21.15.90:0/1889851955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a140b5710 con 0x7f9a140cd8b0 2024-03-16T17:57:24.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.744+0000 7f9a0a000640 1 -- 172.21.15.90:0/1889851955 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9a140b5710 con 0x7f9a140cd8b0 2024-03-16T17:57:24.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.820+0000 7f9a18e00640 1 -- 172.21.15.90:0/1889851955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c948154a-8b9b-4a9a-967a-615651402176", "id": 7} v 0) v1 -- 0x7f9a140afa40 con 0x7f9a140cd8b0 2024-03-16T17:57:24.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.826+0000 7f9a0a000640 1 -- 172.21.15.90:0/1889851955 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c948154a-8b9b-4a9a-967a-615651402176", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f99fc01e460 con 0x7f9a140cd8b0 2024-03-16T17:57:24.830 INFO:teuthology.orchestra.run.smithi090.stdout:7 2024-03-16T17:57:24.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.827+0000 7f9a18e00640 1 -- 172.21.15.90:0/1889851955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a140cd8b0 msgr2=0x7f9a140cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:24.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.827+0000 7f9a18e00640 1 --2- 172.21.15.90:0/1889851955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a140cd8b0 0x7f9a140cba20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00d5f0 tx=0x7f99fc00dab0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.827+0000 7f9a18e00640 1 -- 172.21.15.90:0/1889851955 shutdown_connections 2024-03-16T17:57:24.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.827+0000 7f9a18e00640 1 --2- 172.21.15.90:0/1889851955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9a1415fcc0 0x7f9a140ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.828+0000 7f9a18e00640 1 --2- 172.21.15.90:0/1889851955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a140cd8b0 0x7f9a140cba20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.828+0000 7f9a18e00640 1 --2- 172.21.15.90:0/1889851955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a140b41c0 0x7f9a141e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:24.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.828+0000 7f9a18e00640 1 -- 172.21.15.90:0/1889851955 >> 172.21.15.90:0/1889851955 conn(0x7f9a140b36a0 msgr2=0x7f9a140d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:24.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.828+0000 7f9a18e00640 1 -- 172.21.15.90:0/1889851955 shutdown_connections 2024-03-16T17:57:24.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-03-16T17:57:24.828+0000 7f9a18e00640 1 -- 172.21.15.90:0/1889851955 wait complete. 2024-03-16T17:57:24.849 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-16T17:57:24.849 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-16T17:57:24.856 INFO:tasks.ceph.osd.0:Started 2024-03-16T17:57:24.856 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-16T17:57:24.856 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-16T17:57:24.864 INFO:tasks.ceph.osd.1:Started 2024-03-16T17:57:24.864 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-16T17:57:24.864 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-16T17:57:24.871 INFO:tasks.ceph.osd.2:Started 2024-03-16T17:57:24.871 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-16T17:57:24.872 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-16T17:57:24.876 INFO:tasks.ceph.osd.3:Started 2024-03-16T17:57:24.876 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-16T17:57:24.876 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-16T17:57:24.883 INFO:tasks.ceph.osd.4:Started 2024-03-16T17:57:24.883 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-16T17:57:24.883 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-16T17:57:24.890 INFO:tasks.ceph.osd.5:Started 2024-03-16T17:57:24.891 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-16T17:57:24.891 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-16T17:57:24.898 INFO:tasks.ceph.osd.6:Started 2024-03-16T17:57:24.898 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-16T17:57:24.898 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-16T17:57:24.906 INFO:tasks.ceph.osd.7:Started 2024-03-16T17:57:24.906 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:25.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.124+0000 7f0abaa00640 1 Processor -- start 2024-03-16T17:57:25.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.124+0000 7f0abaa00640 1 -- start start 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.125+0000 7f0abaa00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0ab41454d0 0x7f0ab40b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0abaa00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ab40d0040 0x7f0ab4070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0abaa00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab40b4b90 con 0x7f0ab40cdf80 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0abaa00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab40cf500 con 0x7f0ab41454d0 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0abaa00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41458b0 con 0x7f0ab40d0040 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0ab3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0ab41454d0 0x7f0ab40b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0ab8a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ab40d0040 0x7f0ab4070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0ab3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0ab41454d0 0x7f0ab40b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:56708/0 (socket says 172.21.15.82:56708) 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0ab3e00640 1 -- 172.21.15.82:0/900368185 learned_addr learned my addr 172.21.15.82:0/900368185 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0ab3e00640 1 -- 172.21.15.82:0/900368185 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ab40d0040 msgr2=0x7f0ab4070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:25.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0ab3e00640 1 --2- 172.21.15.82:0/900368185 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ab40d0040 0x7f0ab4070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:25.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0ab3e00640 1 -- 172.21.15.82:0/900368185 >> v1:172.21.15.90:6789/0 conn(0x7f0ab40cdf80 legacy=0x7f0ab40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:25.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.126+0000 7f0ab3e00640 1 -- 172.21.15.82:0/900368185 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40ce460 con 0x7f0ab41454d0 2024-03-16T17:57:25.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.127+0000 7f0ab3e00640 1 --2- 172.21.15.82:0/900368185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ab41454d0 0x7f0ab40b34c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009870 tx=0x7f0aa402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7421c728daaf4c38 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:25.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0ab3400640 1 -- 172.21.15.82:0/900368185 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4030840 con 0x7f0ab41454d0 2024-03-16T17:57:25.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0ab3400640 1 -- 172.21.15.82:0/900368185 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa40309a0 con 0x7f0ab41454d0 2024-03-16T17:57:25.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0ab3400640 1 -- 172.21.15.82:0/900368185 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4030d30 con 0x7f0ab41454d0 2024-03-16T17:57:25.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0abaa00640 1 -- 172.21.15.82:0/900368185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ab41454d0 msgr2=0x7f0ab40b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:25.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0abaa00640 1 --2- 172.21.15.82:0/900368185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ab41454d0 0x7f0ab40b34c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009870 tx=0x7f0aa402b980 comp rx=0 tx=0).stop 2024-03-16T17:57:25.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0abaa00640 1 -- 172.21.15.82:0/900368185 shutdown_connections 2024-03-16T17:57:25.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0abaa00640 1 --2- 172.21.15.82:0/900368185 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ab40d0040 0x7f0ab4070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:25.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0abaa00640 1 --2- 172.21.15.82:0/900368185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ab41454d0 0x7f0ab40b34c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:25.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0abaa00640 1 -- 172.21.15.82:0/900368185 >> 172.21.15.82:0/900368185 conn(0x7f0ab406ae70 msgr2=0x7f0ab406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:25.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0abaa00640 1 -- 172.21.15.82:0/900368185 shutdown_connections 2024-03-16T17:57:25.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.128+0000 7f0abaa00640 1 -- 172.21.15.82:0/900368185 wait complete. 2024-03-16T17:57:25.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.130+0000 7f0abaa00640 1 Processor -- start 2024-03-16T17:57:25.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.130+0000 7f0abaa00640 1 -- start start 2024-03-16T17:57:25.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0abaa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ab40cdf80 0x7f0ab41f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:25.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0abaa00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ab40d0040 0x7f0ab4143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:25.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0abaa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab41454d0 0x7f0ab40aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0abaa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41f4e70 con 0x7f0ab41454d0 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0abaa00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41f4240 con 0x7f0ab40cdf80 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0abaa00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41f4520 con 0x7f0ab40d0040 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0ab8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab41454d0 0x7f0ab40aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0ab8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab41454d0 0x7f0ab40aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34394/0 (socket says 172.21.15.82:34394) 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0ab8a00640 1 -- 172.21.15.82:0/2168682844 learned_addr learned my addr 172.21.15.82:0/2168682844 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0ab3e00640 1 --2- 172.21.15.82:0/2168682844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ab40cdf80 0x7f0ab41f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.131+0000 7f0aabe00640 1 --2- 172.21.15.82:0/2168682844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ab40d0040 0x7f0ab4143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.132+0000 7f0ab8a00640 1 -- 172.21.15.82:0/2168682844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ab40d0040 msgr2=0x7f0ab4143e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.132+0000 7f0ab8a00640 1 --2- 172.21.15.82:0/2168682844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ab40d0040 0x7f0ab4143e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.132+0000 7f0ab8a00640 1 -- 172.21.15.82:0/2168682844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ab40cdf80 msgr2=0x7f0ab41f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.132+0000 7f0ab8a00640 1 --2- 172.21.15.82:0/2168682844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ab40cdf80 0x7f0ab41f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.132+0000 7f0ab8a00640 1 -- 172.21.15.82:0/2168682844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa40094d0 con 0x7f0ab41454d0 2024-03-16T17:57:25.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.132+0000 7f0ab8a00640 1 --2- 172.21.15.82:0/2168682844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab41454d0 0x7f0ab40aeac0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00efc0 tx=0x7f0aac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:25.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.132+0000 7f0ab0c00640 1 -- 172.21.15.82:0/2168682844 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac009ca0 con 0x7f0ab41454d0 2024-03-16T17:57:25.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.132+0000 7f0ab0c00640 1 -- 172.21.15.82:0/2168682844 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac00f040 con 0x7f0ab41454d0 2024-03-16T17:57:25.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.132+0000 7f0abaa00640 1 -- 172.21.15.82:0/2168682844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab41f4800 con 0x7f0ab41454d0 2024-03-16T17:57:25.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.132+0000 7f0ab0c00640 1 -- 172.21.15.82:0/2168682844 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac0146c0 con 0x7f0ab41454d0 2024-03-16T17:57:25.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.133+0000 7f0abaa00640 1 -- 172.21.15.82:0/2168682844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab41f4ae0 con 0x7f0ab41454d0 2024-03-16T17:57:25.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.133+0000 7f0ab0c00640 1 -- 172.21.15.82:0/2168682844 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0aac014bb0 con 0x7f0ab41454d0 2024-03-16T17:57:25.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.133+0000 7f0ab0c00640 1 -- 172.21.15.82:0/2168682844 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0aac013b50 con 0x7f0ab41454d0 2024-03-16T17:57:25.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.134+0000 7f0abaa00640 1 -- 172.21.15.82:0/2168682844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab41f4fe0 con 0x7f0ab41454d0 2024-03-16T17:57:25.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.137+0000 7f0ab0c00640 1 -- 172.21.15.82:0/2168682844 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0aac03bdd0 con 0x7f0ab41454d0 2024-03-16T17:57:25.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.203+0000 7f0abaa00640 1 -- 172.21.15.82:0/2168682844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ab4067f30 con 0x7f0ab41454d0 2024-03-16T17:57:25.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.203+0000 7f0ab0c00640 1 -- 172.21.15.82:0/2168682844 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f0aac02a030 con 0x7f0ab41454d0 2024-03-16T17:57:25.207 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:25.207 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:24.823529+0000","last_up_change":"0.000000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:25.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.204+0000 7f0abaa00640 1 -- 172.21.15.82:0/2168682844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab41454d0 msgr2=0x7f0ab40aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:25.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.204+0000 7f0abaa00640 1 --2- 172.21.15.82:0/2168682844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab41454d0 0x7f0ab40aeac0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00efc0 tx=0x7f0aac00c490 comp rx=0 tx=0).stop 2024-03-16T17:57:25.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.205+0000 7f0abaa00640 1 -- 172.21.15.82:0/2168682844 shutdown_connections 2024-03-16T17:57:25.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.205+0000 7f0abaa00640 1 --2- 172.21.15.82:0/2168682844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ab41454d0 0x7f0ab40aeac0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:25.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.205+0000 7f0abaa00640 1 --2- 172.21.15.82:0/2168682844 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ab40d0040 0x7f0ab4143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:25.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.205+0000 7f0abaa00640 1 --2- 172.21.15.82:0/2168682844 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ab40cdf80 0x7f0ab41f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:25.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.205+0000 7f0abaa00640 1 -- 172.21.15.82:0/2168682844 >> 172.21.15.82:0/2168682844 conn(0x7f0ab406ae70 msgr2=0x7f0ab406db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:25.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.205+0000 7f0abaa00640 1 -- 172.21.15.82:0/2168682844 shutdown_connections 2024-03-16T17:57:25.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:25.205+0000 7f0abaa00640 1 -- 172.21.15.82:0/2168682844 wait complete. 2024-03-16T17:57:25.223 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-16T17:57:25.224 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-16T17:57:25.483 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:25.479+0000 7f3dda63f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T17:57:25.549 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-03-16T17:57:25.545+0000 7f07a621d640 -1 Falling back to public interface 2024-03-16T17:57:25.559 INFO:tasks.ceph.osd.4.smithi090.stderr:2024-03-16T17:57:25.555+0000 7fee7879a640 -1 Falling back to public interface 2024-03-16T17:57:25.572 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-03-16T17:57:25.569+0000 7f47dca48640 -1 Falling back to public interface 2024-03-16T17:57:25.581 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-03-16T17:57:25.578+0000 7f7a0c306640 -1 Falling back to public interface 2024-03-16T17:57:25.586 INFO:tasks.ceph.osd.5.smithi090.stderr:2024-03-16T17:57:25.582+0000 7f7d0a321640 -1 Falling back to public interface 2024-03-16T17:57:25.594 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-03-16T17:57:25.590+0000 7f40481ac640 -1 Falling back to public interface 2024-03-16T17:57:25.609 INFO:tasks.ceph.osd.7.smithi090.stderr:2024-03-16T17:57:25.605+0000 7f3dc7d6d640 -1 Falling back to public interface 2024-03-16T17:57:25.609 INFO:tasks.ceph.osd.6.smithi090.stderr:2024-03-16T17:57:25.605+0000 7f3a75213640 -1 Falling back to public interface 2024-03-16T17:57:25.794 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:25.790+0000 7f3dda63f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T17:57:25.854 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:25.851+0000 7f3dda63f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T17:57:25.875 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:25.872+0000 7f798dcda040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T17:57:25.984 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:25.981+0000 7f3dda63f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T17:57:26.030 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:26.175 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.171+0000 7f1113200640 1 Processor -- start 2024-03-16T17:57:26.175 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.172+0000 7f1113200640 1 -- start start 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1113200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c0d2a90 0x7f110c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1113200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f110c0d0ee0 0x7f110c0b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1113200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110c0d4d30 con 0x7f110c15e350 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1113200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110c0b33c0 con 0x7f110c0d0ee0 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1113200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0b36a0 con 0x7f110c0d2a90 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1111200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f110c0d0ee0 0x7f110c0b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1103e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c0d2a90 0x7f110c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1111200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f110c0d0ee0 0x7f110c0b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34470/0 (socket says 172.21.15.82:34470) 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1111200640 1 -- 172.21.15.82:0/788398259 learned_addr learned my addr 172.21.15.82:0/788398259 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f110be00640 1 --1- 172.21.15.82:0/788398259 >> v1:172.21.15.90:6789/0 conn(0x7f110c15e350 0x7f110c0d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.82:60140/0 (socket says 172.21.15.82:60140) 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1111200640 1 -- 172.21.15.82:0/788398259 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c0d2a90 msgr2=0x7f110c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1111200640 1 --2- 172.21.15.82:0/788398259 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c0d2a90 0x7f110c0b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1111200640 1 -- 172.21.15.82:0/788398259 >> v1:172.21.15.90:6789/0 conn(0x7f110c15e350 legacy=0x7f110c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:26.177 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1111200640 1 -- 172.21.15.82:0/788398259 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c0b3980 con 0x7f110c0d0ee0 2024-03-16T17:57:26.178 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.173+0000 7f1103e00640 1 --2- 172.21.15.82:0/788398259 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c0d2a90 0x7f110c0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.174+0000 7f1111200640 1 --2- 172.21.15.82:0/788398259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 0x7f110c0b46c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1104009e50 tx=0x7f110402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479878dd37ed56eb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f110b400640 1 -- 172.21.15.82:0/788398259 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110402caf0 con 0x7f110c0d0ee0 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f110b400640 1 -- 172.21.15.82:0/788398259 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110402cc50 con 0x7f110c0d0ee0 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f110b400640 1 -- 172.21.15.82:0/788398259 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11040343f0 con 0x7f110c0d0ee0 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f1113200640 1 -- 172.21.15.82:0/788398259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 msgr2=0x7f110c0b46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f1113200640 1 --2- 172.21.15.82:0/788398259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 0x7f110c0b46c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1104009e50 tx=0x7f110402bf50 comp rx=0 tx=0).stop 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f1113200640 1 -- 172.21.15.82:0/788398259 shutdown_connections 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f1113200640 1 --2- 172.21.15.82:0/788398259 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c0d2a90 0x7f110c0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f1113200640 1 --2- 172.21.15.82:0/788398259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 0x7f110c0b46c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f1113200640 1 -- 172.21.15.82:0/788398259 >> 172.21.15.82:0/788398259 conn(0x7f110c0d3da0 msgr2=0x7f110c0d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:26.179 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f1113200640 1 -- 172.21.15.82:0/788398259 shutdown_connections 2024-03-16T17:57:26.180 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.175+0000 7f1113200640 1 -- 172.21.15.82:0/788398259 wait complete. 2024-03-16T17:57:26.180 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.177+0000 7f1113200640 1 Processor -- start 2024-03-16T17:57:26.180 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.177+0000 7f1113200640 1 -- start start 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.178+0000 7f1113200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 0x7f110c1f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.178+0000 7f1113200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f110c0d2a90 0x7f110c14d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.178+0000 7f1113200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c15e350 0x7f110c0b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.178+0000 7f1113200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c1f8000 con 0x7f110c0d0ee0 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.178+0000 7f1113200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c1f4a90 con 0x7f110c0d2a90 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.178+0000 7f1113200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c1f7480 con 0x7f110c15e350 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.178+0000 7f110be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 0x7f110c1f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.178+0000 7f110be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 0x7f110c1f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34484/0 (socket says 172.21.15.82:34484) 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.178+0000 7f110be00640 1 -- 172.21.15.82:0/425287050 learned_addr learned my addr 172.21.15.82:0/425287050 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.178+0000 7f1103e00640 1 --2- 172.21.15.82:0/425287050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f110c0d2a90 0x7f110c14d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:26.183 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f1111200640 1 --2- 172.21.15.82:0/425287050 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c15e350 0x7f110c0b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f110be00640 1 -- 172.21.15.82:0/425287050 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c15e350 msgr2=0x7f110c0b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f110be00640 1 --2- 172.21.15.82:0/425287050 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c15e350 0x7f110c0b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f110be00640 1 -- 172.21.15.82:0/425287050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f110c0d2a90 msgr2=0x7f110c14d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f110be00640 1 --2- 172.21.15.82:0/425287050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f110c0d2a90 0x7f110c14d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f110be00640 1 -- 172.21.15.82:0/425287050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f4009400 con 0x7f110c0d0ee0 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f110be00640 1 --2- 172.21.15.82:0/425287050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 0x7f110c1f4550 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00c3c0 tx=0x7f10fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f1108c00640 1 -- 172.21.15.82:0/425287050 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc0074e0 con 0x7f110c0d0ee0 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f1108c00640 1 -- 172.21.15.82:0/425287050 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc007640 con 0x7f110c0d0ee0 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f1108c00640 1 -- 172.21.15.82:0/425287050 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc004a00 con 0x7f110c0d0ee0 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f1113200640 1 -- 172.21.15.82:0/425287050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1104009b30 con 0x7f110c0d0ee0 2024-03-16T17:57:26.184 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.179+0000 7f1113200640 1 -- 172.21.15.82:0/425287050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c1f7a40 con 0x7f110c0d0ee0 2024-03-16T17:57:26.185 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.180+0000 7f1108c00640 1 -- 172.21.15.82:0/425287050 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 2) v1 ==== 44756+0+0 (secure 0 0 0) 0x7f10fc004b60 con 0x7f110c0d0ee0 2024-03-16T17:57:26.185 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.180+0000 7f1113200640 1 -- 172.21.15.82:0/425287050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110c04f250 con 0x7f110c0d0ee0 2024-03-16T17:57:26.185 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.181+0000 7f1108c00640 1 -- 172.21.15.82:0/425287050 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f10fc0492c0 con 0x7f110c0d0ee0 2024-03-16T17:57:26.186 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.183+0000 7f1108c00640 1 -- 172.21.15.82:0/425287050 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f10fc0192c0 con 0x7f110c0d0ee0 2024-03-16T17:57:26.226 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:26.222+0000 7f798dcda040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T17:57:26.253 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.249+0000 7f1113200640 1 -- 172.21.15.82:0/425287050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f110c0d4d30 con 0x7f110c0d0ee0 2024-03-16T17:57:26.254 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.250+0000 7f1108c00640 1 -- 172.21.15.82:0/425287050 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f110c0d4d30 con 0x7f110c0d0ee0 2024-03-16T17:57:26.254 INFO:teuthology.misc.health.smithi082.stdout: 2024-03-16T17:57:26.254 INFO:teuthology.misc.health.smithi082.stdout:{"epoch":10,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:24.823529+0000","last_up_change":"0.000000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:26.255 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.251+0000 7f1113200640 1 -- 172.21.15.82:0/425287050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 msgr2=0x7f110c1f4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:26.255 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.251+0000 7f1113200640 1 --2- 172.21.15.82:0/425287050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 0x7f110c1f4550 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f10fc00c3c0 tx=0x7f10fc00c880 comp rx=0 tx=0).stop 2024-03-16T17:57:26.255 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.251+0000 7f1113200640 1 -- 172.21.15.82:0/425287050 shutdown_connections 2024-03-16T17:57:26.255 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.252+0000 7f1113200640 1 --2- 172.21.15.82:0/425287050 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f110c15e350 0x7f110c0b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:26.256 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.252+0000 7f1113200640 1 --2- 172.21.15.82:0/425287050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f110c0d2a90 0x7f110c14d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:26.256 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.252+0000 7f1113200640 1 --2- 172.21.15.82:0/425287050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f110c0d0ee0 0x7f110c1f4550 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:26.256 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.252+0000 7f1113200640 1 -- 172.21.15.82:0/425287050 >> 172.21.15.82:0/425287050 conn(0x7f110c0d3da0 msgr2=0x7f110c15d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:26.256 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.252+0000 7f1113200640 1 -- 172.21.15.82:0/425287050 shutdown_connections 2024-03-16T17:57:26.256 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:26.252+0000 7f1113200640 1 -- 172.21.15.82:0/425287050 wait complete. 2024-03-16T17:57:26.268 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-16T17:57:26.286 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:26.283+0000 7f798dcda040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T17:57:26.416 INFO:tasks.ceph.mgr.x.smithi090.stderr:2024-03-16T17:57:26.413+0000 7f798dcda040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T17:57:27.013 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:27.009+0000 7f3d7b000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-16T17:57:27.015 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-03-16T17:57:27.011+0000 7f3d7b000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-16T17:57:29.065 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-03-16T17:57:29.062+0000 7f07a621d640 -1 osd.0 0 log_to_monitors true 2024-03-16T17:57:29.113 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-03-16T17:57:29.110+0000 7f47dca48640 -1 osd.1 0 log_to_monitors true 2024-03-16T17:57:29.116 INFO:tasks.ceph.osd.5.smithi090.stderr:2024-03-16T17:57:29.113+0000 7f7d0a321640 -1 osd.5 0 log_to_monitors true 2024-03-16T17:57:29.145 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-03-16T17:57:29.142+0000 7f7a0c306640 -1 osd.3 0 log_to_monitors true 2024-03-16T17:57:29.149 INFO:tasks.ceph.osd.6.smithi090.stderr:2024-03-16T17:57:29.146+0000 7f3a75213640 -1 osd.6 0 log_to_monitors true 2024-03-16T17:57:29.159 INFO:tasks.ceph.osd.7.smithi090.stderr:2024-03-16T17:57:29.155+0000 7f3dc7d6d640 -1 osd.7 0 log_to_monitors true 2024-03-16T17:57:29.171 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-03-16T17:57:29.168+0000 7f40481ac640 -1 osd.2 0 log_to_monitors true 2024-03-16T17:57:29.321 INFO:tasks.ceph.osd.4.smithi090.stderr:2024-03-16T17:57:29.317+0000 7fee7879a640 -1 osd.4 0 log_to_monitors true 2024-03-16T17:57:32.165 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-03-16T17:57:32.161+0000 7f7a06800640 -1 osd.3 0 waiting for initial osdmap 2024-03-16T17:57:32.170 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-03-16T17:57:32.167+0000 7f79fd200640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T17:57:32.177 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-03-16T17:57:32.174+0000 7f07a2000640 -1 osd.0 0 waiting for initial osdmap 2024-03-16T17:57:32.179 INFO:tasks.ceph.osd.6.smithi090.stderr:2024-03-16T17:57:32.175+0000 7f3a70e00640 -1 osd.6 0 waiting for initial osdmap 2024-03-16T17:57:32.184 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-03-16T17:57:32.181+0000 7f47d6e00640 -1 osd.1 0 waiting for initial osdmap 2024-03-16T17:57:32.184 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-03-16T17:57:32.181+0000 7f0796c00640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T17:57:32.188 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-03-16T17:57:32.185+0000 7f47cd800640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T17:57:32.196 INFO:tasks.ceph.osd.6.smithi090.stderr:2024-03-16T17:57:32.193+0000 7f3a65a00640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T17:57:32.228 INFO:tasks.ceph.osd.5.smithi090.stderr:2024-03-16T17:57:32.225+0000 7f7d06600640 -1 osd.5 0 waiting for initial osdmap 2024-03-16T17:57:32.237 INFO:tasks.ceph.osd.5.smithi090.stderr:2024-03-16T17:57:32.233+0000 7f7cfb200640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T17:57:32.280 INFO:tasks.ceph.osd.7.smithi090.stderr:2024-03-16T17:57:32.276+0000 7f3dc2200640 -1 osd.7 0 waiting for initial osdmap 2024-03-16T17:57:32.289 INFO:tasks.ceph.osd.7.smithi090.stderr:2024-03-16T17:57:32.285+0000 7f3db8c00640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T17:57:32.662 INFO:tasks.ceph.osd.4.smithi090.stderr:2024-03-16T17:57:32.658+0000 7fee72c00640 -1 osd.4 0 waiting for initial osdmap 2024-03-16T17:57:32.669 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-03-16T17:57:32.667+0000 7f4042600640 -1 osd.2 0 waiting for initial osdmap 2024-03-16T17:57:32.671 INFO:tasks.ceph.osd.4.smithi090.stderr:2024-03-16T17:57:32.667+0000 7fee69600640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T17:57:32.678 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-03-16T17:57:32.676+0000 7f4039000640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T17:57:33.074 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:33.276 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.272+0000 7f01ab800640 1 Processor -- start 2024-03-16T17:57:33.276 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.272+0000 7f01ab800640 1 -- start start 2024-03-16T17:57:33.278 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.273+0000 7f01ab800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f01a40cc310 0x7f01a40cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.278 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.273+0000 7f01ab800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f01a40cf3d0 0x7f01a415cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.278 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.273+0000 7f01ab800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a40b3f10 0x7f01a40b4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.278 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.273+0000 7f01ab800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a40ce000 con 0x7f01a40cf3d0 2024-03-16T17:57:33.278 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.273+0000 7f01ab800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a40b48b0 con 0x7f01a40cc310 2024-03-16T17:57:33.278 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.273+0000 7f01ab800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a415d330 con 0x7f01a40b3f10 2024-03-16T17:57:33.278 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.274+0000 7f01a3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f01a40cf3d0 0x7f01a415cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.278 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.274+0000 7f01a3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f01a40cf3d0 0x7f01a415cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34540/0 (socket says 172.21.15.82:34540) 2024-03-16T17:57:33.278 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.274+0000 7f01a3e00640 1 -- 172.21.15.82:0/769794813 learned_addr learned my addr 172.21.15.82:0/769794813 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:33.278 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.274+0000 7f01a8e00640 1 --2- 172.21.15.82:0/769794813 >> v2:172.21.15.90:3300/0 conn(0x7f01a40cc310 0x7f01a40cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.279 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.274+0000 7f01a9800640 1 --2- 172.21.15.82:0/769794813 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a40b3f10 0x7f01a40b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.279 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.274+0000 7f01a3e00640 1 -- 172.21.15.82:0/769794813 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a40b3f10 msgr2=0x7f01a40b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:33.279 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.274+0000 7f01a3e00640 1 --2- 172.21.15.82:0/769794813 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a40b3f10 0x7f01a40b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.279 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.274+0000 7f01a3e00640 1 -- 172.21.15.82:0/769794813 >> v2:172.21.15.90:3300/0 conn(0x7f01a40cc310 msgr2=0x7f01a40cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:33.279 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.274+0000 7f01a3e00640 1 --2- 172.21.15.82:0/769794813 >> v2:172.21.15.90:3300/0 conn(0x7f01a40cc310 0x7f01a40cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.279 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.274+0000 7f01a3e00640 1 -- 172.21.15.82:0/769794813 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a40cfb30 con 0x7f01a40cf3d0 2024-03-16T17:57:33.280 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.275+0000 7f01a3e00640 1 --2- 172.21.15.82:0/769794813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 0x7f01a415cdf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01a40cfd20 tx=0x7f019002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b671328cac484b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:33.280 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.275+0000 7f01a3400640 1 -- 172.21.15.82:0/769794813 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01900317c0 con 0x7f01a40cf3d0 2024-03-16T17:57:33.280 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.275+0000 7f01a3400640 1 -- 172.21.15.82:0/769794813 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0190031920 con 0x7f01a40cf3d0 2024-03-16T17:57:33.280 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.275+0000 7f01a3400640 1 -- 172.21.15.82:0/769794813 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190031cb0 con 0x7f01a40cf3d0 2024-03-16T17:57:33.280 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.276+0000 7f01ab800640 1 -- 172.21.15.82:0/769794813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 msgr2=0x7f01a415cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.280 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.276+0000 7f01ab800640 1 --2- 172.21.15.82:0/769794813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 0x7f01a415cdf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01a40cfd20 tx=0x7f019002b860 comp rx=0 tx=0).stop 2024-03-16T17:57:33.280 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.276+0000 7f01ab800640 1 -- 172.21.15.82:0/769794813 shutdown_connections 2024-03-16T17:57:33.280 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.276+0000 7f01ab800640 1 --2- 172.21.15.82:0/769794813 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a40b3f10 0x7f01a40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.281 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.276+0000 7f01ab800640 1 --2- 172.21.15.82:0/769794813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 0x7f01a415cdf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.281 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.276+0000 7f01ab800640 1 --2- 172.21.15.82:0/769794813 >> v2:172.21.15.90:3300/0 conn(0x7f01a40cc310 0x7f01a40cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.281 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.276+0000 7f01ab800640 1 -- 172.21.15.82:0/769794813 >> 172.21.15.82:0/769794813 conn(0x7f01a406adb0 msgr2=0x7f01a406b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:33.281 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.276+0000 7f01ab800640 1 -- 172.21.15.82:0/769794813 shutdown_connections 2024-03-16T17:57:33.281 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.276+0000 7f01ab800640 1 -- 172.21.15.82:0/769794813 wait complete. 2024-03-16T17:57:33.281 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.278+0000 7f01ab800640 1 Processor -- start 2024-03-16T17:57:33.281 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.278+0000 7f01ab800640 1 -- start start 2024-03-16T17:57:33.283 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01ab800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a40b3f10 0x7f01a4142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01ab800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a40cc310 0x7f01a4158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01ab800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 0x7f01a4159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01ab800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a415baa0 con 0x7f01a40cf3d0 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01ab800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4145590 con 0x7f01a40cc310 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01ab800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a4145840 con 0x7f01a40b3f10 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 0x7f01a4159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 0x7f01a4159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34548/0 (socket says 172.21.15.82:34548) 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a9800640 1 -- 172.21.15.82:0/262971924 learned_addr learned my addr 172.21.15.82:0/262971924 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a3e00640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a40cc310 0x7f01a4158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a8e00640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a40b3f10 0x7f01a4142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a9800640 1 -- 172.21.15.82:0/262971924 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a40b3f10 msgr2=0x7f01a4142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a9800640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a40b3f10 0x7f01a4142530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.284 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a9800640 1 -- 172.21.15.82:0/262971924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a40cc310 msgr2=0x7f01a4158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.285 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a9800640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a40cc310 0x7f01a4158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.285 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a9800640 1 -- 172.21.15.82:0/262971924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900094d0 con 0x7f01a40cf3d0 2024-03-16T17:57:33.285 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.279+0000 7f01a3e00640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a40cc310 0x7f01a4158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:33.285 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.280+0000 7f01a9800640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 0x7f01a4159100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f019c00ccf0 tx=0x7f019c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:33.285 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.280+0000 7f01a0c00640 1 -- 172.21.15.82:0/262971924 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c003b90 con 0x7f01a40cf3d0 2024-03-16T17:57:33.285 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.280+0000 7f01a0c00640 1 -- 172.21.15.82:0/262971924 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019c003cf0 con 0x7f01a40cf3d0 2024-03-16T17:57:33.285 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.280+0000 7f01ab800640 1 -- 172.21.15.82:0/262971924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a4152ed0 con 0x7f01a40cf3d0 2024-03-16T17:57:33.285 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.280+0000 7f01a0c00640 1 -- 172.21.15.82:0/262971924 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c004240 con 0x7f01a40cf3d0 2024-03-16T17:57:33.285 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.280+0000 7f01ab800640 1 -- 172.21.15.82:0/262971924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a4153490 con 0x7f01a40cf3d0 2024-03-16T17:57:33.286 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.281+0000 7f01ab800640 1 -- 172.21.15.82:0/262971924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a404f250 con 0x7f01a40cf3d0 2024-03-16T17:57:33.286 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.282+0000 7f01a0c00640 1 -- 172.21.15.82:0/262971924 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) v1 ==== 89406+0+0 (secure 0 0 0) 0x7f019c0043a0 con 0x7f01a40cf3d0 2024-03-16T17:57:33.286 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.283+0000 7f01a0c00640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f018406b320 0x7f018406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.286 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.283+0000 7f01a0c00640 1 -- 172.21.15.82:0/262971924 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f019c012070 con 0x7f01a40cf3d0 2024-03-16T17:57:33.286 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.283+0000 7f01a8e00640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f018406b320 0x7f018406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.287 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.284+0000 7f01a8e00640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f018406b320 0x7f018406d7e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0198006fd0 tx=0x7f0198008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:33.291 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.288+0000 7f01a0c00640 1 -- 172.21.15.82:0/262971924 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f019c0551b0 con 0x7f01a40cf3d0 2024-03-16T17:57:33.477 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.474+0000 7f01ab800640 1 -- 172.21.15.82:0/262971924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01a40b37a0 con 0x7f01a40cf3d0 2024-03-16T17:57:33.479 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.475+0000 7f01a0c00640 1 -- 172.21.15.82:0/262971924 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10873 (secure 0 0 0) 0x7f019c058870 con 0x7f01a40cf3d0 2024-03-16T17:57:33.479 INFO:teuthology.misc.health.smithi082.stdout: 2024-03-16T17:57:33.479 INFO:teuthology.misc.health.smithi082.stdout:{"epoch":14,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:33.170235+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 -- 172.21.15.82:0/262971924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f018406b320 msgr2=0x7f018406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f018406b320 0x7f018406d7e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0198006fd0 tx=0x7f0198008040 comp rx=0 tx=0).stop 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 -- 172.21.15.82:0/262971924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 msgr2=0x7f01a4159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 0x7f01a4159100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f019c00ccf0 tx=0x7f019c00a660 comp rx=0 tx=0).stop 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 -- 172.21.15.82:0/262971924 shutdown_connections 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f018406b320 0x7f018406d7e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01a40cf3d0 0x7f01a4159100 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01a40cc310 0x7f01a4158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 --2- 172.21.15.82:0/262971924 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f01a40b3f10 0x7f01a4142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 -- 172.21.15.82:0/262971924 >> 172.21.15.82:0/262971924 conn(0x7f01a406adb0 msgr2=0x7f01a415aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 -- 172.21.15.82:0/262971924 shutdown_connections 2024-03-16T17:57:33.482 INFO:teuthology.misc.health.smithi082.stderr:2024-03-16T17:57:33.478+0000 7f01ab800640 1 -- 172.21.15.82:0/262971924 wait complete. 2024-03-16T17:57:33.494 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-16T17:57:33.494 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-16T17:57:33.495 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-16T17:57:33.495 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-16T17:57:33.500 INFO:tasks.ceph.mds.a:Started 2024-03-16T17:57:33.500 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-16T17:57:33.500 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-16T17:57:33.505 INFO:tasks.ceph.mds.c:Started 2024-03-16T17:57:33.505 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-16T17:57:33.505 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-16T17:57:33.513 INFO:tasks.ceph.mds.b:Started 2024-03-16T17:57:33.513 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-16T17:57:33.513 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-16T17:57:33.520 INFO:tasks.ceph.mds.d:Started 2024-03-16T17:57:33.521 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-16T17:57:33.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:33.647 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-03-16T17:57:33.649 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-03-16T17:57:33.667 INFO:tasks.ceph.mds.b.smithi090.stdout:starting mds.b at 2024-03-16T17:57:33.674 INFO:tasks.ceph.mds.d.smithi090.stdout:starting mds.d at 2024-03-16T17:57:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.701+0000 7fb783e00640 1 Processor -- start 2024-03-16T17:57:33.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.701+0000 7fb783e00640 1 -- start start 2024-03-16T17:57:33.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.702+0000 7fb783e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb784070820 0x7fb78415f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.702+0000 7fb783e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb784143e00 0x7fb784070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.702+0000 7fb783e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7840cf3a0 con 0x7fb78415b090 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.702+0000 7fb783e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb78415f5c0 con 0x7fb784070820 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.702+0000 7fb783e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb784071140 con 0x7fb784143e00 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb782a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb784070820 0x7fb78415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb782a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb784070820 0x7fb78415f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34588/0 (socket says 172.21.15.82:34588) 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb783400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb78415b090 0x7fb78415b520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45440/0 (socket says 172.21.15.82:45440) 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb782a00640 1 -- 172.21.15.82:0/3512227750 learned_addr learned my addr 172.21.15.82:0/3512227750 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb782000640 1 --2- 172.21.15.82:0/3512227750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb784143e00 0x7fb784070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb782a00640 1 -- 172.21.15.82:0/3512227750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb784143e00 msgr2=0x7fb784070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb782a00640 1 --2- 172.21.15.82:0/3512227750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb784143e00 0x7fb784070c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb782a00640 1 -- 172.21.15.82:0/3512227750 >> v1:172.21.15.82:6789/0 conn(0x7fb78415b090 legacy=0x7fb78415b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb782a00640 1 -- 172.21.15.82:0/3512227750 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7840cf610 con 0x7fb784070820 2024-03-16T17:57:33.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb781600640 1 -- 172.21.15.82:0/3512227750 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995996234 0 0) 0x7fb7840cf3a0 con 0x7fb78415b090 2024-03-16T17:57:33.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.703+0000 7fb782000640 1 --2- 172.21.15.82:0/3512227750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb784143e00 0x7fb784070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.704+0000 7fb782a00640 1 --2- 172.21.15.82:0/3512227750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 0x7fb78415f080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb774009870 tx=0x7fb77402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d23d83c4432c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.704+0000 7fb781600640 1 -- 172.21.15.82:0/3512227750 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774031640 con 0x7fb784070820 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.704+0000 7fb781600640 1 -- 172.21.15.82:0/3512227750 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7740317a0 con 0x7fb784070820 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.704+0000 7fb781600640 1 -- 172.21.15.82:0/3512227750 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774031b30 con 0x7fb784070820 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.705+0000 7fb783e00640 1 -- 172.21.15.82:0/3512227750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 msgr2=0x7fb78415f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.705+0000 7fb783e00640 1 --2- 172.21.15.82:0/3512227750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 0x7fb78415f080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb774009870 tx=0x7fb77402b8f0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.705+0000 7fb783e00640 1 -- 172.21.15.82:0/3512227750 shutdown_connections 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.705+0000 7fb783e00640 1 --2- 172.21.15.82:0/3512227750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb784143e00 0x7fb784070c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.705+0000 7fb783e00640 1 --2- 172.21.15.82:0/3512227750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 0x7fb78415f080 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.705+0000 7fb783e00640 1 -- 172.21.15.82:0/3512227750 >> 172.21.15.82:0/3512227750 conn(0x7fb78406ae70 msgr2=0x7fb78406b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:33.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.705+0000 7fb783e00640 1 -- 172.21.15.82:0/3512227750 shutdown_connections 2024-03-16T17:57:33.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.705+0000 7fb783e00640 1 -- 172.21.15.82:0/3512227750 wait complete. 2024-03-16T17:57:33.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.707+0000 7fb783e00640 1 Processor -- start 2024-03-16T17:57:33.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.707+0000 7fb783e00640 1 -- start start 2024-03-16T17:57:33.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb783e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 0x7fb784159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb783e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb784143e00 0x7fb7841e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb783e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb78415b090 0x7fb7841e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb783e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784130040 con 0x7fb784070820 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb783e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7841e55a0 con 0x7fb784143e00 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb783e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78415a320 con 0x7fb78415b090 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb782a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 0x7fb784159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb782a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 0x7fb784159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34602/0 (socket says 172.21.15.82:34602) 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb782a00640 1 -- 172.21.15.82:0/798540417 learned_addr learned my addr 172.21.15.82:0/798540417 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb782000640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb784143e00 0x7fb7841e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb782a00640 1 -- 172.21.15.82:0/798540417 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb78415b090 msgr2=0x7fb7841e57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb782a00640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb78415b090 0x7fb7841e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb782a00640 1 -- 172.21.15.82:0/798540417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb784143e00 msgr2=0x7fb7841e44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb782a00640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb784143e00 0x7fb7841e44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb782a00640 1 -- 172.21.15.82:0/798540417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740094d0 con 0x7fb784070820 2024-03-16T17:57:33.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.708+0000 7fb782000640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb784143e00 0x7fb7841e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:33.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.709+0000 7fb782a00640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 0x7fb784159de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb774000c00 tx=0x7fb7740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:33.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.709+0000 7fb77b400640 1 -- 172.21.15.82:0/798540417 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77403ac60 con 0x7fb784070820 2024-03-16T17:57:33.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.709+0000 7fb77b400640 1 -- 172.21.15.82:0/798540417 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77403ade0 con 0x7fb784070820 2024-03-16T17:57:33.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.709+0000 7fb77b400640 1 -- 172.21.15.82:0/798540417 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774044630 con 0x7fb784070820 2024-03-16T17:57:33.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.709+0000 7fb783e00640 1 -- 172.21.15.82:0/798540417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7841e4d00 con 0x7fb784070820 2024-03-16T17:57:33.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.709+0000 7fb783e00640 1 -- 172.21.15.82:0/798540417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7841e52c0 con 0x7fb784070820 2024-03-16T17:57:33.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.710+0000 7fb783e00640 1 -- 172.21.15.82:0/798540417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7840b4d80 con 0x7fb784070820 2024-03-16T17:57:33.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.711+0000 7fb77b400640 1 -- 172.21.15.82:0/798540417 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) v1 ==== 89406+0+0 (secure 0 0 0) 0x7fb77402f050 con 0x7fb784070820 2024-03-16T17:57:33.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.712+0000 7fb77b400640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb76406b320 0x7fb76406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:33.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.712+0000 7fb77b400640 1 -- 172.21.15.82:0/798540417 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fb7740ab6e0 con 0x7fb784070820 2024-03-16T17:57:33.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.712+0000 7fb782000640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb76406b320 0x7fb76406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:33.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.713+0000 7fb782000640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb76406b320 0x7fb76406d7e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb770006fd0 tx=0x7fb770008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:33.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.717+0000 7fb77b400640 1 -- 172.21.15.82:0/798540417 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb774076990 con 0x7fb784070820 2024-03-16T17:57:33.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.914+0000 7fb783e00640 1 -- 172.21.15.82:0/798540417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb78404f250 con 0x7fb784070820 2024-03-16T17:57:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.915+0000 7fb77b400640 1 -- 172.21.15.82:0/798540417 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10873 (secure 0 0 0) 0x7fb77407a050 con 0x7fb784070820 2024-03-16T17:57:33.919 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:33.919 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":14,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:33.170235+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 -- 172.21.15.82:0/798540417 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb76406b320 msgr2=0x7fb76406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb76406b320 0x7fb76406d7e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb770006fd0 tx=0x7fb770008040 comp rx=0 tx=0).stop 2024-03-16T17:57:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 -- 172.21.15.82:0/798540417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 msgr2=0x7fb784159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 0x7fb784159de0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb774000c00 tx=0x7fb7740046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 -- 172.21.15.82:0/798540417 shutdown_connections 2024-03-16T17:57:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb76406b320 0x7fb76406d7e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb78415b090 0x7fb7841e57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb784143e00 0x7fb7841e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 --2- 172.21.15.82:0/798540417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb784070820 0x7fb784159de0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 -- 172.21.15.82:0/798540417 >> 172.21.15.82:0/798540417 conn(0x7fb78406ae70 msgr2=0x7fb78415a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:33.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 -- 172.21.15.82:0/798540417 shutdown_connections 2024-03-16T17:57:33.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:33.918+0000 7fb783e00640 1 -- 172.21.15.82:0/798540417 wait complete. 2024-03-16T17:57:33.938 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-03-16T17:57:33.938 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:57:34.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.131+0000 7f1c82800640 1 Processor -- start 2024-03-16T17:57:34.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.131+0000 7f1c82800640 1 -- start start 2024-03-16T17:57:34.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.132+0000 7f1c82800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1c7c1454d0 0x7f1c7c0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:34.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.132+0000 7f1c82800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7c0cdf80 0x7f1c7c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:34.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.132+0000 7f1c82800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c7c0cfe70 0x7f1c7c15f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:34.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.132+0000 7f1c82800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c0d0320 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.132+0000 7f1c82800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c7c0b4b90 con 0x7f1c7c1454d0 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.132+0000 7f1c82800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0cf500 con 0x7f1c7c0cdf80 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c7be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c7c0cfe70 0x7f1c7c15f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c7be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1c7c0cfe70 0x7f1c7c15f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34616/0 (socket says 172.21.15.82:34616) 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c7be00640 1 -- 172.21.15.82:0/2325263543 learned_addr learned my addr 172.21.15.82:0/2325263543 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c7b400640 1 --2- 172.21.15.82:0/2325263543 >> v2:172.21.15.90:3300/0 conn(0x7f1c7c1454d0 0x7f1c7c0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c73e00640 1 --2- 172.21.15.82:0/2325263543 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7c0cdf80 0x7f1c7c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c7be00640 1 -- 172.21.15.82:0/2325263543 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7c0cdf80 msgr2=0x7f1c7c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c7be00640 1 --2- 172.21.15.82:0/2325263543 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7c0cdf80 0x7f1c7c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c7be00640 1 -- 172.21.15.82:0/2325263543 >> v2:172.21.15.90:3300/0 conn(0x7f1c7c1454d0 msgr2=0x7f1c7c0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c7be00640 1 --2- 172.21.15.82:0/2325263543 >> v2:172.21.15.90:3300/0 conn(0x7f1c7c1454d0 0x7f1c7c0b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c7be00640 1 -- 172.21.15.82:0/2325263543 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c1458b0 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.133+0000 7f1c7b400640 1 --2- 172.21.15.82:0/2325263543 >> v2:172.21.15.90:3300/0 conn(0x7f1c7c1454d0 0x7f1c7c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.135+0000 7f1c7be00640 1 --2- 172.21.15.82:0/2325263543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 0x7f1c7c15f8b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0b2800 tx=0x7f1c7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef0e0a07494da59 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.135+0000 7f1c7aa00640 1 -- 172.21.15.82:0/2325263543 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c7402ca70 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.135+0000 7f1c7aa00640 1 -- 172.21.15.82:0/2325263543 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7402cbd0 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.135+0000 7f1c7aa00640 1 -- 172.21.15.82:0/2325263543 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740343f0 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.136+0000 7f1c82800640 1 -- 172.21.15.82:0/2325263543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 msgr2=0x7f1c7c15f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.136+0000 7f1c82800640 1 --2- 172.21.15.82:0/2325263543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 0x7f1c7c15f8b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c0b2800 tx=0x7f1c7402bed0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.136+0000 7f1c82800640 1 -- 172.21.15.82:0/2325263543 shutdown_connections 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.136+0000 7f1c82800640 1 --2- 172.21.15.82:0/2325263543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 0x7f1c7c15f8b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.136+0000 7f1c82800640 1 --2- 172.21.15.82:0/2325263543 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7c0cdf80 0x7f1c7c0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.136+0000 7f1c82800640 1 --2- 172.21.15.82:0/2325263543 >> v2:172.21.15.90:3300/0 conn(0x7f1c7c1454d0 0x7f1c7c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.136+0000 7f1c82800640 1 -- 172.21.15.82:0/2325263543 >> 172.21.15.82:0/2325263543 conn(0x7f1c7c06ae70 msgr2=0x7f1c7c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.136+0000 7f1c82800640 1 -- 172.21.15.82:0/2325263543 shutdown_connections 2024-03-16T17:57:34.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.136+0000 7f1c82800640 1 -- 172.21.15.82:0/2325263543 wait complete. 2024-03-16T17:57:34.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.137+0000 7f1c82800640 1 Processor -- start 2024-03-16T17:57:34.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.138+0000 7f1c82800640 1 -- start start 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.138+0000 7f1c82800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c7c0cdf80 0x7f1c7c0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.138+0000 7f1c82800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 0x7f1c7c1f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.138+0000 7f1c82800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7c1454d0 0x7f1c7c1f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.138+0000 7f1c82800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0af720 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c82800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0af970 con 0x7f1c7c0cdf80 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c82800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7c0afc50 con 0x7f1c7c1454d0 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c73e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 0x7f1c7c1f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c73e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 0x7f1c7c1f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34628/0 (socket says 172.21.15.82:34628) 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c73e00640 1 -- 172.21.15.82:0/183972473 learned_addr learned my addr 172.21.15.82:0/183972473 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c7b400640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c7c0cdf80 0x7f1c7c0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:34.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c7be00640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7c1454d0 0x7f1c7c1f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:34.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c73e00640 1 -- 172.21.15.82:0/183972473 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7c1454d0 msgr2=0x7f1c7c1f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:34.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c73e00640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7c1454d0 0x7f1c7c1f4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c73e00640 1 -- 172.21.15.82:0/183972473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c7c0cdf80 msgr2=0x7f1c7c0b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:34.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c73e00640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c7c0cdf80 0x7f1c7c0b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c73e00640 1 -- 172.21.15.82:0/183972473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c74009b30 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c7b400640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c7c0cdf80 0x7f1c7c0b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:34.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.139+0000 7f1c73e00640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 0x7f1c7c1f2820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400d800 tx=0x7f1c6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:34.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.140+0000 7f1c73400640 1 -- 172.21.15.82:0/183972473 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c640096a0 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.140+0000 7f1c73400640 1 -- 172.21.15.82:0/183972473 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c64009800 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.140+0000 7f1c73400640 1 -- 172.21.15.82:0/183972473 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64009990 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.140+0000 7f1c82800640 1 -- 172.21.15.82:0/183972473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7c0aff30 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.140+0000 7f1c82800640 1 -- 172.21.15.82:0/183972473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7c0b24a0 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.141+0000 7f1c82800640 1 -- 172.21.15.82:0/183972473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7c067f30 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.142+0000 7f1c73400640 1 -- 172.21.15.82:0/183972473 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) v1 ==== 89406+0+0 (secure 0 0 0) 0x7f1c64009af0 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.143+0000 7f1c73400640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1c5806b320 0x7f1c5806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:34.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.143+0000 7f1c73400640 1 -- 172.21.15.82:0/183972473 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f1c640899e0 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.143+0000 7f1c7b400640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1c5806b320 0x7f1c5806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:34.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.144+0000 7f1c7b400640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1c5806b320 0x7f1c5806d7e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c15c550 tx=0x7f1c6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:34.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.148+0000 7f1c73400640 1 -- 172.21.15.82:0/183972473 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1c64054c80 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.384+0000 7f1c82800640 1 -- 172.21.15.82:0/183972473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c7c143d10 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.385+0000 7f1c73400640 1 -- 172.21.15.82:0/183972473 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f1c64018540 con 0x7f1c7c0cfe70 2024-03-16T17:57:34.389 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:34.389 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3721205060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3721205060},{"type":"v1","addr":"172.21.15.82:6835","nonce":3721205060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-16T17:57:34.389 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 2 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 -- 172.21.15.82:0/183972473 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1c5806b320 msgr2=0x7f1c5806d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1c5806b320 0x7f1c5806d7e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c15c550 tx=0x7f1c6c008040 comp rx=0 tx=0).stop 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 -- 172.21.15.82:0/183972473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 msgr2=0x7f1c7c1f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 0x7f1c7c1f2820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400d800 tx=0x7f1c6400dcc0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 -- 172.21.15.82:0/183972473 shutdown_connections 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1c5806b320 0x7f1c5806d7e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1c7c1454d0 0x7f1c7c1f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c7c0cfe70 0x7f1c7c1f2820 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 --2- 172.21.15.82:0/183972473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1c7c0cdf80 0x7f1c7c0b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 -- 172.21.15.82:0/183972473 >> 172.21.15.82:0/183972473 conn(0x7f1c7c06ae70 msgr2=0x7f1c7c13f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 -- 172.21.15.82:0/183972473 shutdown_connections 2024-03-16T17:57:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:34.388+0000 7f1c82800640 1 -- 172.21.15.82:0/183972473 wait complete. 2024-03-16T17:57:35.216 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:57:35.223 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:57:36.410 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:57:36.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.607+0000 7fa897000640 1 Processor -- start 2024-03-16T17:57:36.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.607+0000 7fa897000640 1 -- start start 2024-03-16T17:57:36.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.608+0000 7fa897000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa89015f550 0x7fa89015ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:36.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa897000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89015e820 con 0x7fa890144400 2024-03-16T17:57:36.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa897000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89015c080 con 0x7fa8900d00a0 2024-03-16T17:57:36.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa897000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89015f130 con 0x7fa89015f550 2024-03-16T17:57:36.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa88fe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa890144400 0x7fa8900cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48516/0 (socket says 172.21.15.82:48516) 2024-03-16T17:57:36.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa88fe00640 1 -- 172.21.15.82:0/1031276776 learned_addr learned my addr 172.21.15.82:0/1031276776 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:36.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa895000640 1 --2- 172.21.15.82:0/1031276776 >> v2:172.21.15.82:3300/0 conn(0x7fa89015f550 0x7fa89015ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:36.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa88f400640 1 -- 172.21.15.82:0/1031276776 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564072473 0 0) 0x7fa89015e820 con 0x7fa890144400 2024-03-16T17:57:36.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa88f400640 1 -- 172.21.15.82:0/1031276776 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa874003520 con 0x7fa890144400 2024-03-16T17:57:36.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa895000640 1 -- 172.21.15.82:0/1031276776 >> v1:172.21.15.90:6789/0 conn(0x7fa8900d00a0 legacy=0x7fa89015b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:36.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa895000640 1 -- 172.21.15.82:0/1031276776 >> v1:172.21.15.82:6789/0 conn(0x7fa890144400 legacy=0x7fa8900cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:36.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.609+0000 7fa895000640 1 -- 172.21.15.82:0/1031276776 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89015ea70 con 0x7fa89015f550 2024-03-16T17:57:36.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.611+0000 7fa895000640 1 --2- 172.21.15.82:0/1031276776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 0x7fa89015ba20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa888009d90 tx=0x7fa88802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758178cb383ece1a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:36.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.611+0000 7fa88f400640 1 -- 172.21.15.82:0/1031276776 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88802cc60 con 0x7fa89015f550 2024-03-16T17:57:36.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.611+0000 7fa88f400640 1 -- 172.21.15.82:0/1031276776 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88802cdc0 con 0x7fa89015f550 2024-03-16T17:57:36.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.611+0000 7fa88f400640 1 -- 172.21.15.82:0/1031276776 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888034640 con 0x7fa89015f550 2024-03-16T17:57:36.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.612+0000 7fa897000640 1 -- 172.21.15.82:0/1031276776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 msgr2=0x7fa89015ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:36.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.612+0000 7fa897000640 1 --2- 172.21.15.82:0/1031276776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 0x7fa89015ba20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa888009d90 tx=0x7fa88802beb0 comp rx=0 tx=0).stop 2024-03-16T17:57:36.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.612+0000 7fa897000640 1 -- 172.21.15.82:0/1031276776 shutdown_connections 2024-03-16T17:57:36.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.612+0000 7fa897000640 1 --2- 172.21.15.82:0/1031276776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 0x7fa89015ba20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:36.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.612+0000 7fa897000640 1 -- 172.21.15.82:0/1031276776 >> 172.21.15.82:0/1031276776 conn(0x7fa89006ae70 msgr2=0x7fa89006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:36.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.612+0000 7fa897000640 1 -- 172.21.15.82:0/1031276776 shutdown_connections 2024-03-16T17:57:36.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.612+0000 7fa897000640 1 -- 172.21.15.82:0/1031276776 wait complete. 2024-03-16T17:57:36.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.614+0000 7fa897000640 1 Processor -- start 2024-03-16T17:57:36.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.614+0000 7fa897000640 1 -- start start 2024-03-16T17:57:36.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.615+0000 7fa897000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8900d00a0 0x7fa890141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:36.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.615+0000 7fa897000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa890144400 0x7fa8900b0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:36.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.615+0000 7fa897000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 0x7fa89015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.615+0000 7fa897000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900b19a0 con 0x7fa89015f550 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.615+0000 7fa897000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900b0930 con 0x7fa890144400 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.615+0000 7fa897000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900b0c10 con 0x7fa8900d00a0 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.615+0000 7fa895000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 0x7fa89015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.615+0000 7fa895000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 0x7fa89015c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39392/0 (socket says 172.21.15.82:39392) 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.615+0000 7fa895000640 1 -- 172.21.15.82:0/2770418331 learned_addr learned my addr 172.21.15.82:0/2770418331 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.616+0000 7fa88fe00640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8900d00a0 0x7fa890141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.616+0000 7fa887400640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa890144400 0x7fa8900b0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.616+0000 7fa895000640 1 -- 172.21.15.82:0/2770418331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8900d00a0 msgr2=0x7fa890141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.616+0000 7fa895000640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8900d00a0 0x7fa890141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.616+0000 7fa895000640 1 -- 172.21.15.82:0/2770418331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa890144400 msgr2=0x7fa8900b0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.616+0000 7fa895000640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa890144400 0x7fa8900b0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.616+0000 7fa895000640 1 -- 172.21.15.82:0/2770418331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa888009a70 con 0x7fa89015f550 2024-03-16T17:57:36.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.616+0000 7fa895000640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 0x7fa89015c210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa888009d90 tx=0x7fa888004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:36.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.616+0000 7fa88cc00640 1 -- 172.21.15.82:0/2770418331 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888003b90 con 0x7fa89015f550 2024-03-16T17:57:36.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.617+0000 7fa88cc00640 1 -- 172.21.15.82:0/2770418331 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888003cf0 con 0x7fa89015f550 2024-03-16T17:57:36.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.617+0000 7fa88cc00640 1 -- 172.21.15.82:0/2770418331 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888004270 con 0x7fa89015f550 2024-03-16T17:57:36.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.617+0000 7fa897000640 1 -- 172.21.15.82:0/2770418331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8900b0ef0 con 0x7fa89015f550 2024-03-16T17:57:36.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.617+0000 7fa897000640 1 -- 172.21.15.82:0/2770418331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa890141980 con 0x7fa89015f550 2024-03-16T17:57:36.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.618+0000 7fa897000640 1 -- 172.21.15.82:0/2770418331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8901f3e70 con 0x7fa89015f550 2024-03-16T17:57:36.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.619+0000 7fa88cc00640 1 -- 172.21.15.82:0/2770418331 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa8880043d0 con 0x7fa89015f550 2024-03-16T17:57:36.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.621+0000 7fa88cc00640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa86c06b3e0 0x7fa86c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:36.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.621+0000 7fa88cc00640 1 -- 172.21.15.82:0/2770418331 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa8880ac170 con 0x7fa89015f550 2024-03-16T17:57:36.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.621+0000 7fa88fe00640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa86c06b3e0 0x7fa86c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:36.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.621+0000 7fa88fe00640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa86c06b3e0 0x7fa86c06d8a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa8800011e0 tx=0x7fa880009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:36.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.626+0000 7fa88cc00640 1 -- 172.21.15.82:0/2770418331 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa8901f3e70 con 0x7fa89015f550 2024-03-16T17:57:36.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.867+0000 7fa897000640 1 -- 172.21.15.82:0/2770418331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8901f3e70 con 0x7fa89015f550 2024-03-16T17:57:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.868+0000 7fa88cc00640 1 -- 172.21.15.82:0/2770418331 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3691 (secure 0 0 0) 0x7fa888044e80 con 0x7fa89015f550 2024-03-16T17:57:36.872 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:36.872 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2570794110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2570794110},{"type":"v1","addr":"172.21.15.90:6835","nonce":2570794110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3135739259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3135739259},{"type":"v1","addr":"172.21.15.82:6837","nonce":3135739259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3721205060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3721205060},{"type":"v1","addr":"172.21.15.82:6835","nonce":3721205060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6833/732041173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":732041173},{"type":"v1","addr":"172.21.15.90:6833","nonce":732041173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-16T17:57:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 3 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 -- 172.21.15.82:0/2770418331 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa86c06b3e0 msgr2=0x7fa86c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa86c06b3e0 0x7fa86c06d8a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa8800011e0 tx=0x7fa880009040 comp rx=0 tx=0).stop 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 -- 172.21.15.82:0/2770418331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 msgr2=0x7fa89015c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 0x7fa89015c210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa888009d90 tx=0x7fa888004a40 comp rx=0 tx=0).stop 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 -- 172.21.15.82:0/2770418331 shutdown_connections 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa86c06b3e0 0x7fa86c06d8a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa89015f550 0x7fa89015c210 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa890144400 0x7fa8900b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 --2- 172.21.15.82:0/2770418331 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8900d00a0 0x7fa890141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 -- 172.21.15.82:0/2770418331 >> 172.21.15.82:0/2770418331 conn(0x7fa89006ae70 msgr2=0x7fa890159080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 -- 172.21.15.82:0/2770418331 shutdown_connections 2024-03-16T17:57:36.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:36.871+0000 7fa897000640 1 -- 172.21.15.82:0/2770418331 wait complete. 2024-03-16T17:57:36.892 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:37.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.081+0000 7f18ef200640 1 Processor -- start 2024-03-16T17:57:37.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.081+0000 7f18ef200640 1 -- start start 2024-03-16T17:57:37.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.082+0000 7f18ef200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f18e81454d0 0x7f18e80b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.083+0000 7f18ef200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e80b4b90 con 0x7f18e80cdf80 2024-03-16T17:57:37.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.083+0000 7f18ef200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e80cf500 con 0x7f18e80d0040 2024-03-16T17:57:37.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.083+0000 7f18ef200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e81458b0 con 0x7f18e81454d0 2024-03-16T17:57:37.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.083+0000 7f18e7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f18e81454d0 0x7f18e80b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.083+0000 7f18e7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f18e81454d0 0x7f18e80b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39398/0 (socket says 172.21.15.82:39398) 2024-03-16T17:57:37.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.083+0000 7f18e7e00640 1 -- 172.21.15.82:0/2418891958 learned_addr learned my addr 172.21.15.82:0/2418891958 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:37.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.083+0000 7f18e7e00640 1 -- 172.21.15.82:0/2418891958 >> v1:172.21.15.90:6789/0 conn(0x7f18e80d0040 legacy=0x7f18e8070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.083+0000 7f18e7e00640 1 -- 172.21.15.82:0/2418891958 >> v1:172.21.15.82:6789/0 conn(0x7f18e80cdf80 legacy=0x7f18e80cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.084+0000 7f18e7e00640 1 -- 172.21.15.82:0/2418891958 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e80ce460 con 0x7f18e81454d0 2024-03-16T17:57:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.084+0000 7f18e6a00640 1 -- 172.21.15.82:0/2418891958 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 302329556 0 0) 0x7f18e80b4b90 con 0x7f18e80cdf80 2024-03-16T17:57:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.085+0000 7f18e7e00640 1 --2- 172.21.15.82:0/2418891958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e81454d0 0x7f18e80b34c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009870 tx=0x7f18d802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ddfda878faec59 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.085+0000 7f18e6a00640 1 -- 172.21.15.82:0/2418891958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d802f840 con 0x7f18e81454d0 2024-03-16T17:57:37.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.085+0000 7f18e6a00640 1 -- 172.21.15.82:0/2418891958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d802f9a0 con 0x7f18e81454d0 2024-03-16T17:57:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.085+0000 7f18e6a00640 1 -- 172.21.15.82:0/2418891958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d802fd30 con 0x7f18e81454d0 2024-03-16T17:57:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.085+0000 7f18ef200640 1 -- 172.21.15.82:0/2418891958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e81454d0 msgr2=0x7f18e80b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.085+0000 7f18ef200640 1 --2- 172.21.15.82:0/2418891958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e81454d0 0x7f18e80b34c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f18d8009870 tx=0x7f18d802ba20 comp rx=0 tx=0).stop 2024-03-16T17:57:37.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.086+0000 7f18ef200640 1 -- 172.21.15.82:0/2418891958 shutdown_connections 2024-03-16T17:57:37.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.086+0000 7f18ef200640 1 --2- 172.21.15.82:0/2418891958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e81454d0 0x7f18e80b34c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.086+0000 7f18ef200640 1 -- 172.21.15.82:0/2418891958 >> 172.21.15.82:0/2418891958 conn(0x7f18e806ae70 msgr2=0x7f18e806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:37.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.086+0000 7f18ef200640 1 -- 172.21.15.82:0/2418891958 shutdown_connections 2024-03-16T17:57:37.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.086+0000 7f18ef200640 1 -- 172.21.15.82:0/2418891958 wait complete. 2024-03-16T17:57:37.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.087+0000 7f18ef200640 1 Processor -- start 2024-03-16T17:57:37.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.087+0000 7f18ef200640 1 -- start start 2024-03-16T17:57:37.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.088+0000 7f18ef200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e80cdf80 0x7f18e8154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.088+0000 7f18ef200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18e80d0040 0x7f18e813fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.088+0000 7f18ef200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f18e81454d0 0x7f18e8140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.088+0000 7f18ef200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e813b7b0 con 0x7f18e80cdf80 2024-03-16T17:57:37.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.088+0000 7f18ef200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8142940 con 0x7f18e80d0040 2024-03-16T17:57:37.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.088+0000 7f18ef200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8142c20 con 0x7f18e81454d0 2024-03-16T17:57:37.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18e7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e80cdf80 0x7f18e8154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18e7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e80cdf80 0x7f18e8154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39402/0 (socket says 172.21.15.82:39402) 2024-03-16T17:57:37.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18e7e00640 1 -- 172.21.15.82:0/27377879 learned_addr learned my addr 172.21.15.82:0/27377879 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:37.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18e7400640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18e80d0040 0x7f18e813fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18ed200640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f18e81454d0 0x7f18e8140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18e7e00640 1 -- 172.21.15.82:0/27377879 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f18e81454d0 msgr2=0x7f18e8140150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18e7e00640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f18e81454d0 0x7f18e8140150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18e7e00640 1 -- 172.21.15.82:0/27377879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18e80d0040 msgr2=0x7f18e813fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18e7e00640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18e80d0040 0x7f18e813fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18e7e00640 1 -- 172.21.15.82:0/27377879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d80094d0 con 0x7f18e80cdf80 2024-03-16T17:57:37.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.089+0000 7f18e7e00640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e80cdf80 0x7f18e8154ba0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18d80090b0 tx=0x7f18d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:37.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.090+0000 7f18dfe00640 1 -- 172.21.15.82:0/27377879 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d803c040 con 0x7f18e80cdf80 2024-03-16T17:57:37.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.090+0000 7f18dfe00640 1 -- 172.21.15.82:0/27377879 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d8037d50 con 0x7f18e80cdf80 2024-03-16T17:57:37.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.090+0000 7f18dfe00640 1 -- 172.21.15.82:0/27377879 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8004270 con 0x7f18e80cdf80 2024-03-16T17:57:37.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.090+0000 7f18ef200640 1 -- 172.21.15.82:0/27377879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e8142f00 con 0x7f18e80cdf80 2024-03-16T17:57:37.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.090+0000 7f18ef200640 1 -- 172.21.15.82:0/27377879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e81434c0 con 0x7f18e80cdf80 2024-03-16T17:57:37.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.091+0000 7f18ef200640 1 -- 172.21.15.82:0/27377879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e8067f30 con 0x7f18e80cdf80 2024-03-16T17:57:37.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.092+0000 7f18dfe00640 1 -- 172.21.15.82:0/27377879 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f18d80043d0 con 0x7f18e80cdf80 2024-03-16T17:57:37.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.092+0000 7f18dfe00640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f18c406b3b0 0x7f18c406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.092+0000 7f18dfe00640 1 -- 172.21.15.82:0/27377879 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f18d80aba60 con 0x7f18e80cdf80 2024-03-16T17:57:37.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.093+0000 7f18e7400640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f18c406b3b0 0x7f18c406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.093+0000 7f18e7400640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f18c406b3b0 0x7f18c406d870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f18e8071b60 tx=0x7f18d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:37.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.097+0000 7f18dfe00640 1 -- 172.21.15.82:0/27377879 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f18d8076aa0 con 0x7f18e80cdf80 2024-03-16T17:57:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.301+0000 7f18ef200640 1 -- 172.21.15.82:0/27377879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18e813c4f0 con 0x7f18e80cdf80 2024-03-16T17:57:37.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.303+0000 7f18dfe00640 1 -- 172.21.15.82:0/27377879 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12848 (secure 0 0 0) 0x7f18d8047070 con 0x7f18e80cdf80 2024-03-16T17:57:37.307 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:37.307 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:36.352062+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 -- 172.21.15.82:0/27377879 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f18c406b3b0 msgr2=0x7f18c406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f18c406b3b0 0x7f18c406d870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f18e8071b60 tx=0x7f18d0009040 comp rx=0 tx=0).stop 2024-03-16T17:57:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 -- 172.21.15.82:0/27377879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e80cdf80 msgr2=0x7f18e8154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e80cdf80 0x7f18e8154ba0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18d80090b0 tx=0x7f18d8004730 comp rx=0 tx=0).stop 2024-03-16T17:57:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 -- 172.21.15.82:0/27377879 shutdown_connections 2024-03-16T17:57:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f18c406b3b0 0x7f18c406d870 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f18e81454d0 0x7f18e8140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18e80d0040 0x7f18e813fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 --2- 172.21.15.82:0/27377879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18e80cdf80 0x7f18e8154ba0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 -- 172.21.15.82:0/27377879 >> 172.21.15.82:0/27377879 conn(0x7f18e806ae70 msgr2=0x7f18e80b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:37.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 -- 172.21.15.82:0/27377879 shutdown_connections 2024-03-16T17:57:37.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.306+0000 7f18ef200640 1 -- 172.21.15.82:0/27377879 wait complete. 2024-03-16T17:57:37.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:57:37.327 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:57:37.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.520+0000 7f1273e00640 1 Processor -- start 2024-03-16T17:57:37.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.520+0000 7f1273e00640 1 -- start start 2024-03-16T17:57:37.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.521+0000 7f1273e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127406e650 0x7f1274160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1273e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12741458b0 con 0x7f12741454d0 2024-03-16T17:57:37.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1273e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12740cf3b0 con 0x7f127406e010 2024-03-16T17:57:37.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1273e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12740cf690 con 0x7f127406e650 2024-03-16T17:57:37.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1272a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f12741454d0 0x7f127406d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48524/0 (socket says 172.21.15.82:48524) 2024-03-16T17:57:37.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1272a00640 1 -- 172.21.15.82:0/1355838124 learned_addr learned my addr 172.21.15.82:0/1355838124 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:37.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578709803 0 0) 0x7f12741458b0 con 0x7f12741454d0 2024-03-16T17:57:37.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1258003520 con 0x7f12741454d0 2024-03-16T17:57:37.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1273400640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127406e650 0x7f1274160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1627391338 0 0) 0x7f1258003520 con 0x7f12741454d0 2024-03-16T17:57:37.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127406e650 msgr2=0x7f1274160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:37.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1272000640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127406e650 0x7f1274160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 >> v1:172.21.15.90:6789/0 conn(0x7f127406e010 legacy=0x7f12740b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127415f7a0 con 0x7f12741454d0 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.522+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f1264001670 con 0x7f12741454d0 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 >> v1:172.21.15.82:6789/0 conn(0x7f12741454d0 legacy=0x7f127406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272000640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 0x7f1258003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272000640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127406e650 0x7f12580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272000640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1258004d50 0x7f12580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12740cf690 con 0x7f1258003a10 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125800bb40 con 0x7f1258004d50 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125800bd00 con 0x7f127406e650 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272a00640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 0x7f1258003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1273400640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1258004d50 0x7f12580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f126a000640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127406e650 0x7f12580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272a00640 1 -- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127406e650 msgr2=0x7f12580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272a00640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127406e650 0x7f12580047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272a00640 1 -- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1258004d50 msgr2=0x7f12580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:37.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272a00640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1258004d50 0x7f12580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.523+0000 7f1272a00640 1 -- 172.21.15.82:0/1355838124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258003520 con 0x7f1258003a10 2024-03-16T17:57:37.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.524+0000 7f1272a00640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 0x7f1258003e30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f126400c240 tx=0x7f12580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7216a0108f2a86 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:37.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.525+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1264005ad0 con 0x7f1258003a10 2024-03-16T17:57:37.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.525+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1264005c30 con 0x7f1258003a10 2024-03-16T17:57:37.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.525+0000 7f1272000640 1 -- 172.21.15.82:0/1355838124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126402c8b0 con 0x7f1258003a10 2024-03-16T17:57:37.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.525+0000 7f1273e00640 1 -- 172.21.15.82:0/1355838124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 msgr2=0x7f1258003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.525+0000 7f1273e00640 1 --2- 172.21.15.82:0/1355838124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 0x7f1258003e30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f126400c240 tx=0x7f12580057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.525+0000 7f1273400640 1 -- 172.21.15.82:0/1355838124 reap_dead start 2024-03-16T17:57:37.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.525+0000 7f1273e00640 1 -- 172.21.15.82:0/1355838124 shutdown_connections 2024-03-16T17:57:37.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.525+0000 7f1273e00640 1 -- 172.21.15.82:0/1355838124 >> 172.21.15.82:0/1355838124 conn(0x7f127406ae70 msgr2=0x7f127406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:37.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.525+0000 7f1273e00640 1 -- 172.21.15.82:0/1355838124 shutdown_connections 2024-03-16T17:57:37.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.525+0000 7f1273e00640 1 -- 172.21.15.82:0/1355838124 wait complete. 2024-03-16T17:57:37.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.527+0000 7f1273e00640 1 Processor -- start 2024-03-16T17:57:37.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.527+0000 7f1273e00640 1 -- start start 2024-03-16T17:57:37.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.528+0000 7f1273e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 0x7f1274140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.528+0000 7f1273e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f127415b1e0 0x7f127413c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.528+0000 7f1273e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127415ac70 0x7f127413d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.528+0000 7f1273e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1274140660 con 0x7f1258003a10 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.528+0000 7f1273e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12740b3390 con 0x7f127415b1e0 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.528+0000 7f1273e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12740b2780 con 0x7f127415ac70 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.528+0000 7f1272a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 0x7f1274140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.528+0000 7f1272a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 0x7f1274140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39422/0 (socket says 172.21.15.82:39422) 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.528+0000 7f1272a00640 1 -- 172.21.15.82:0/4235283291 learned_addr learned my addr 172.21.15.82:0/4235283291 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f1273400640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127415ac70 0x7f127413d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f126a000640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f127415b1e0 0x7f127413c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f1272a00640 1 -- 172.21.15.82:0/4235283291 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127415ac70 msgr2=0x7f127413d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f1272a00640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127415ac70 0x7f127413d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f1272a00640 1 -- 172.21.15.82:0/4235283291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f127415b1e0 msgr2=0x7f127413c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f1272a00640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f127415b1e0 0x7f127413c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f1272a00640 1 -- 172.21.15.82:0/4235283291 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12740b2a60 con 0x7f1258003a10 2024-03-16T17:57:37.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f1272a00640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 0x7f1274140b00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f126400c360 tx=0x7f1264001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:37.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f126b400640 1 -- 172.21.15.82:0/4235283291 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126403c040 con 0x7f1258003a10 2024-03-16T17:57:37.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f126b400640 1 -- 172.21.15.82:0/4235283291 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126402ccc0 con 0x7f1258003a10 2024-03-16T17:57:37.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f126b400640 1 -- 172.21.15.82:0/4235283291 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1264007290 con 0x7f1258003a10 2024-03-16T17:57:37.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.529+0000 7f1273e00640 1 -- 172.21.15.82:0/4235283291 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12740b2d40 con 0x7f1258003a10 2024-03-16T17:57:37.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.530+0000 7f1273e00640 1 -- 172.21.15.82:0/4235283291 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1274141040 con 0x7f1258003a10 2024-03-16T17:57:37.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.531+0000 7f1273e00640 1 -- 172.21.15.82:0/4235283291 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127404f250 con 0x7f1258003a10 2024-03-16T17:57:37.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.531+0000 7f126b400640 1 -- 172.21.15.82:0/4235283291 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f1264032070 con 0x7f1258003a10 2024-03-16T17:57:37.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.532+0000 7f126b400640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f125006b3e0 0x7f125006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.532+0000 7f126b400640 1 -- 172.21.15.82:0/4235283291 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f12640aa8d0 con 0x7f1258003a10 2024-03-16T17:57:37.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.533+0000 7f126a000640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f125006b3e0 0x7f125006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.533+0000 7f126a000640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f125006b3e0 0x7f125006d8a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1274140a90 tx=0x7f125c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:37.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.537+0000 7f126b400640 1 -- 172.21.15.82:0/4235283291 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1264075890 con 0x7f1258003a10 2024-03-16T17:57:37.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.731+0000 7f1273e00640 1 -- 172.21.15.82:0/4235283291 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f127406e430 con 0x7f1258003a10 2024-03-16T17:57:37.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.732+0000 7f126b400640 1 -- 172.21.15.82:0/4235283291 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1264046070 con 0x7f1258003a10 2024-03-16T17:57:37.736 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:57:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 -- 172.21.15.82:0/4235283291 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f125006b3e0 msgr2=0x7f125006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f125006b3e0 0x7f125006d8a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1274140a90 tx=0x7f125c009040 comp rx=0 tx=0).stop 2024-03-16T17:57:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 -- 172.21.15.82:0/4235283291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 msgr2=0x7f1274140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 0x7f1274140b00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f126400c360 tx=0x7f1264001a00 comp rx=0 tx=0).stop 2024-03-16T17:57:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 -- 172.21.15.82:0/4235283291 shutdown_connections 2024-03-16T17:57:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f125006b3e0 0x7f125006d8a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f127415ac70 0x7f127413d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f127415b1e0 0x7f127413c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 --2- 172.21.15.82:0/4235283291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1258003a10 0x7f1274140b00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 -- 172.21.15.82:0/4235283291 >> 172.21.15.82:0/4235283291 conn(0x7f127406ae70 msgr2=0x7f12740cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:37.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 -- 172.21.15.82:0/4235283291 shutdown_connections 2024-03-16T17:57:37.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.735+0000 7f1273e00640 1 -- 172.21.15.82:0/4235283291 wait complete. 2024-03-16T17:57:37.757 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-16T17:57:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.961+0000 7f8422c00640 1 Processor -- start 2024-03-16T17:57:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.961+0000 7f8422c00640 1 -- start start 2024-03-16T17:57:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.963+0000 7f8422c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f841c15dee0 0x7f841c0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.963+0000 7f8422c00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f841c15e2c0 0x7f841c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.963+0000 7f8422c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841c06d8c0 con 0x7f841c072310 2024-03-16T17:57:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.963+0000 7f8422c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841c15e790 con 0x7f841c15dee0 2024-03-16T17:57:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.963+0000 7f8422c00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841c15ea70 con 0x7f841c15e2c0 2024-03-16T17:57:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.963+0000 7f841be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f841c15dee0 0x7f841c0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.963+0000 7f841be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f841c15dee0 0x7f841c0b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39438/0 (socket says 172.21.15.82:39438) 2024-03-16T17:57:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.963+0000 7f841be00640 1 -- 172.21.15.82:0/1773380834 learned_addr learned my addr 172.21.15.82:0/1773380834 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.963+0000 7f841b400640 1 --2- 172.21.15.82:0/1773380834 >> v2:172.21.15.90:3300/0 conn(0x7f841c15e2c0 0x7f841c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.964+0000 7f841be00640 1 -- 172.21.15.82:0/1773380834 >> v2:172.21.15.90:3300/0 conn(0x7f841c15e2c0 msgr2=0x7f841c06d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.964+0000 7f841be00640 1 --2- 172.21.15.82:0/1773380834 >> v2:172.21.15.90:3300/0 conn(0x7f841c15e2c0 0x7f841c06d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.964+0000 7f841be00640 1 -- 172.21.15.82:0/1773380834 >> v1:172.21.15.82:6789/0 conn(0x7f841c072310 legacy=0x7f841c0ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.964+0000 7f841be00640 1 -- 172.21.15.82:0/1773380834 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841c15ed50 con 0x7f841c15dee0 2024-03-16T17:57:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.965+0000 7f841be00640 1 --2- 172.21.15.82:0/1773380834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c15dee0 0x7f841c0b48b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f841c0d0600 tx=0x7f840c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd35279384ac550 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:37.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.965+0000 7f841aa00640 1 -- 172.21.15.82:0/1773380834 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c004230 con 0x7f841c15dee0 2024-03-16T17:57:37.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.965+0000 7f841aa00640 1 -- 172.21.15.82:0/1773380834 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c02e050 con 0x7f841c15dee0 2024-03-16T17:57:37.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.965+0000 7f841aa00640 1 -- 172.21.15.82:0/1773380834 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c003610 con 0x7f841c15dee0 2024-03-16T17:57:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.966+0000 7f8422c00640 1 -- 172.21.15.82:0/1773380834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c15dee0 msgr2=0x7f841c0b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.966+0000 7f8422c00640 1 --2- 172.21.15.82:0/1773380834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c15dee0 0x7f841c0b48b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f841c0d0600 tx=0x7f840c02b7e0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.966+0000 7f8422c00640 1 -- 172.21.15.82:0/1773380834 shutdown_connections 2024-03-16T17:57:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.966+0000 7f8422c00640 1 --2- 172.21.15.82:0/1773380834 >> v2:172.21.15.90:3300/0 conn(0x7f841c15e2c0 0x7f841c06d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.966+0000 7f8422c00640 1 --2- 172.21.15.82:0/1773380834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c15dee0 0x7f841c0b48b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.966+0000 7f8422c00640 1 -- 172.21.15.82:0/1773380834 >> 172.21.15.82:0/1773380834 conn(0x7f841c06ae70 msgr2=0x7f841c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.966+0000 7f8422c00640 1 -- 172.21.15.82:0/1773380834 shutdown_connections 2024-03-16T17:57:37.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.966+0000 7f8422c00640 1 -- 172.21.15.82:0/1773380834 wait complete. 2024-03-16T17:57:37.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.968+0000 7f8422c00640 1 Processor -- start 2024-03-16T17:57:37.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.968+0000 7f8422c00640 1 -- start start 2024-03-16T17:57:37.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f8422c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c072310 0x7f841c14a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f8422c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f841c15dee0 0x7f841c0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f8422c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f841c15e2c0 0x7f841c0b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f8422c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c0b1240 con 0x7f841c072310 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f8422c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841c14ae20 con 0x7f841c15dee0 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f8422c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841c14b100 con 0x7f841c15e2c0 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f841be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c072310 0x7f841c14a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f841be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c072310 0x7f841c14a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39446/0 (socket says 172.21.15.82:39446) 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f841be00640 1 -- 172.21.15.82:0/1869582576 learned_addr learned my addr 172.21.15.82:0/1869582576 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f841b400640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f841c15dee0 0x7f841c0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f8420c00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f841c15e2c0 0x7f841c0b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f841be00640 1 -- 172.21.15.82:0/1869582576 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f841c15e2c0 msgr2=0x7f841c0b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f841be00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f841c15e2c0 0x7f841c0b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f841be00640 1 -- 172.21.15.82:0/1869582576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f841c15dee0 msgr2=0x7f841c0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:37.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f841be00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f841c15dee0 0x7f841c0b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:37.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f841be00640 1 -- 172.21.15.82:0/1869582576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c0094d0 con 0x7f841c072310 2024-03-16T17:57:37.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.969+0000 7f8420c00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f841c15e2c0 0x7f841c0b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:37.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.970+0000 7f841be00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c072310 0x7f841c14a660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f840c0090b0 tx=0x7f840c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:37.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.970+0000 7f8413e00640 1 -- 172.21.15.82:0/1869582576 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c003e50 con 0x7f841c072310 2024-03-16T17:57:37.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.970+0000 7f8413e00640 1 -- 172.21.15.82:0/1869582576 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c030510 con 0x7f841c072310 2024-03-16T17:57:37.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.970+0000 7f8413e00640 1 -- 172.21.15.82:0/1869582576 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c030880 con 0x7f841c072310 2024-03-16T17:57:37.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.970+0000 7f8422c00640 1 -- 172.21.15.82:0/1869582576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841c14b3e0 con 0x7f841c072310 2024-03-16T17:57:37.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.970+0000 7f8422c00640 1 -- 172.21.15.82:0/1869582576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f841c0c4a20 con 0x7f841c072310 2024-03-16T17:57:37.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.971+0000 7f8422c00640 1 -- 172.21.15.82:0/1869582576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841c0c4bb0 con 0x7f841c072310 2024-03-16T17:57:37.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.973+0000 7f8413e00640 1 -- 172.21.15.82:0/1869582576 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f840c039040 con 0x7f841c072310 2024-03-16T17:57:37.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.973+0000 7f8413e00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83fc06b3b0 0x7f83fc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:37.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.973+0000 7f8413e00640 1 -- 172.21.15.82:0/1869582576 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f840c0aafa0 con 0x7f841c072310 2024-03-16T17:57:37.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.974+0000 7f841b400640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83fc06b3b0 0x7f83fc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:37.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.974+0000 7f841b400640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83fc06b3b0 0x7f83fc06d870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f841c15fd00 tx=0x7f8408009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:37.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:37.978+0000 7f8413e00640 1 -- 172.21.15.82:0/1869582576 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f840c075ff0 con 0x7f841c072310 2024-03-16T17:57:38.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.180+0000 7f8422c00640 1 -- 172.21.15.82:0/1869582576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f841c067f30 con 0x7f841c072310 2024-03-16T17:57:38.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.302+0000 7f8413e00640 1 -- 172.21.15.82:0/1869582576 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f840c04c090 con 0x7f841c072310 2024-03-16T17:57:38.306 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:38.306 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-03-16T17:57:38.306 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:57:38.306 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-03-16T17:57:38.306 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-03-16T17:57:38.306 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:57:38.307 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-03-16T17:57:38.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.305+0000 7f8422c00640 1 -- 172.21.15.82:0/1869582576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83fc06b3b0 msgr2=0x7f83fc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:38.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.305+0000 7f8422c00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83fc06b3b0 0x7f83fc06d870 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f841c15fd00 tx=0x7f8408009190 comp rx=0 tx=0).stop 2024-03-16T17:57:38.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.305+0000 7f8422c00640 1 -- 172.21.15.82:0/1869582576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c072310 msgr2=0x7f841c14a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:38.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.305+0000 7f8422c00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c072310 0x7f841c14a660 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f840c0090b0 tx=0x7f840c0046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.305+0000 7f8422c00640 1 -- 172.21.15.82:0/1869582576 shutdown_connections 2024-03-16T17:57:38.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.305+0000 7f8422c00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83fc06b3b0 0x7f83fc06d870 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.305+0000 7f8422c00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f841c15e2c0 0x7f841c0b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.305+0000 7f8422c00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f841c15dee0 0x7f841c0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.305+0000 7f8422c00640 1 --2- 172.21.15.82:0/1869582576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f841c072310 0x7f841c14a660 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.305+0000 7f8422c00640 1 -- 172.21.15.82:0/1869582576 >> 172.21.15.82:0/1869582576 conn(0x7f841c06ae70 msgr2=0x7f841c06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:38.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.306+0000 7f8422c00640 1 -- 172.21.15.82:0/1869582576 shutdown_connections 2024-03-16T17:57:38.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.306+0000 7f8422c00640 1 -- 172.21.15.82:0/1869582576 wait complete. 2024-03-16T17:57:38.329 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-16T17:57:38.329 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-16T17:57:38.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.532+0000 7f1a4bc00640 1 Processor -- start 2024-03-16T17:57:38.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.532+0000 7f1a4bc00640 1 -- start start 2024-03-16T17:57:38.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.533+0000 7f1a4bc00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1a4415e410 0x7f1a440cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:38.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.533+0000 7f1a4bc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a440cf770 con 0x7f1a440d2390 2024-03-16T17:57:38.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.533+0000 7f1a4bc00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a440ce8c0 con 0x7f1a4415e030 2024-03-16T17:57:38.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.533+0000 7f1a4bc00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a440ceb60 con 0x7f1a4415e410 2024-03-16T17:57:38.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.534+0000 7f1a49c00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1a440d2390 0x7f1a440c7900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48546/0 (socket says 172.21.15.82:48546) 2024-03-16T17:57:38.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.534+0000 7f1a49c00640 1 -- 172.21.15.82:0/1060467434 learned_addr learned my addr 172.21.15.82:0/1060467434 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.534+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301198358 0 0) 0x7f1a440cf770 con 0x7f1a440d2390 2024-03-16T17:57:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.534+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a28003520 con 0x7f1a440d2390 2024-03-16T17:57:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.534+0000 7f1a43e00640 1 --2- 172.21.15.82:0/1060467434 >> v2:172.21.15.90:3300/0 conn(0x7f1a4415e410 0x7f1a440cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3862776966 0 0) 0x7f1a28003520 con 0x7f1a440d2390 2024-03-16T17:57:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 >> v2:172.21.15.90:3300/0 conn(0x7f1a4415e410 msgr2=0x7f1a440cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 --2- 172.21.15.82:0/1060467434 >> v2:172.21.15.90:3300/0 conn(0x7f1a4415e410 0x7f1a440cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 >> v1:172.21.15.90:6789/0 conn(0x7f1a4415e030 legacy=0x7f1a440d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:38.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440cee40 con 0x7f1a440d2390 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f1a3c001670 con 0x7f1a440d2390 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 >> v1:172.21.15.82:6789/0 conn(0x7f1a440d2390 legacy=0x7f1a440c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a28003970 0x7f1a28003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1a280047c0 0x7f1a28004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a280052a0 0x7f1a2800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440ceb60 con 0x7f1a28003970 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2800c1a0 con 0x7f1a280052a0 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2800c360 con 0x7f1a280047c0 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a49200640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a28003970 0x7f1a28003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a49c00640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a280052a0 0x7f1a2800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a49200640 1 -- 172.21.15.82:0/1060467434 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1a280047c0 msgr2=0x7f1a28004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a43e00640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1a280047c0 0x7f1a28004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a49200640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1a280047c0 0x7f1a28004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a49200640 1 -- 172.21.15.82:0/1060467434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a280052a0 msgr2=0x7f1a2800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a49200640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a280052a0 0x7f1a2800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a49200640 1 -- 172.21.15.82:0/1060467434 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a28003520 con 0x7f1a28003970 2024-03-16T17:57:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a49c00640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a280052a0 0x7f1a2800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.535+0000 7f1a49c00640 1 -- 172.21.15.82:0/1060467434 reap_dead start 2024-03-16T17:57:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.536+0000 7f1a49200640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a28003970 0x7f1a28003d70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a34001d20 tx=0x7f1a280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62777fa11cdbf4c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.537+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34003b90 con 0x7f1a28003970 2024-03-16T17:57:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.537+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34003d10 con 0x7f1a28003970 2024-03-16T17:57:38.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.537+0000 7f1a43400640 1 -- 172.21.15.82:0/1060467434 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a340052b0 con 0x7f1a28003970 2024-03-16T17:57:38.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.537+0000 7f1a4bc00640 1 -- 172.21.15.82:0/1060467434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a28003970 msgr2=0x7f1a28003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:38.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.537+0000 7f1a4bc00640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a28003970 0x7f1a28003d70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1a34001d20 tx=0x7f1a280057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.537+0000 7f1a4bc00640 1 -- 172.21.15.82:0/1060467434 shutdown_connections 2024-03-16T17:57:38.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.537+0000 7f1a4bc00640 1 --2- 172.21.15.82:0/1060467434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a28003970 0x7f1a28003d70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.537+0000 7f1a4bc00640 1 -- 172.21.15.82:0/1060467434 >> 172.21.15.82:0/1060467434 conn(0x7f1a440d3db0 msgr2=0x7f1a440d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:38.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.537+0000 7f1a4bc00640 1 -- 172.21.15.82:0/1060467434 shutdown_connections 2024-03-16T17:57:38.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.537+0000 7f1a4bc00640 1 -- 172.21.15.82:0/1060467434 wait complete. 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.539+0000 7f1a4bc00640 1 Processor -- start 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.539+0000 7f1a4bc00640 1 -- start start 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a4bc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a28003970 0x7f1a440cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a4bc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a440cb940 0x7f1a44159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a4bc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1a44156e30 0x7f1a4415bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a4bc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440cdb60 con 0x7f1a440cb940 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a4bc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44157ef0 con 0x7f1a28003970 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a4bc00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44160a00 con 0x7f1a44156e30 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a43e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a440cb940 0x7f1a44159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a43e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a440cb940 0x7f1a44159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39488/0 (socket says 172.21.15.82:39488) 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a43e00640 1 -- 172.21.15.82:0/3888728471 learned_addr learned my addr 172.21.15.82:0/3888728471 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a49c00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1a44156e30 0x7f1a4415bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.540+0000 7f1a49200640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a28003970 0x7f1a440cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:38.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a43e00640 1 -- 172.21.15.82:0/3888728471 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1a44156e30 msgr2=0x7f1a4415bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:38.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a43e00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1a44156e30 0x7f1a4415bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a43e00640 1 -- 172.21.15.82:0/3888728471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a28003970 msgr2=0x7f1a440cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:38.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a43e00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a28003970 0x7f1a440cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:38.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a43e00640 1 -- 172.21.15.82:0/3888728471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a44160ce0 con 0x7f1a440cb940 2024-03-16T17:57:38.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a43e00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a440cb940 0x7f1a44159c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00d5e0 tx=0x7f1a2c00daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:38.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a40c00640 1 -- 172.21.15.82:0/3888728471 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c004440 con 0x7f1a440cb940 2024-03-16T17:57:38.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a40c00640 1 -- 172.21.15.82:0/3888728471 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a2c00c040 con 0x7f1a440cb940 2024-03-16T17:57:38.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a40c00640 1 -- 172.21.15.82:0/3888728471 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c003850 con 0x7f1a440cb940 2024-03-16T17:57:38.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a4bc00640 1 -- 172.21.15.82:0/3888728471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a44160fc0 con 0x7f1a440cb940 2024-03-16T17:57:38.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.541+0000 7f1a4bc00640 1 -- 172.21.15.82:0/3888728471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a44161580 con 0x7f1a440cb940 2024-03-16T17:57:38.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.542+0000 7f1a4bc00640 1 -- 172.21.15.82:0/3888728471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a44158060 con 0x7f1a440cb940 2024-03-16T17:57:38.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.543+0000 7f1a40c00640 1 -- 172.21.15.82:0/3888728471 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f1a2c0039b0 con 0x7f1a440cb940 2024-03-16T17:57:38.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.544+0000 7f1a40c00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1a2006b3b0 0x7f1a2006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:38.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.544+0000 7f1a40c00640 1 -- 172.21.15.82:0/3888728471 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1a2c089ba0 con 0x7f1a440cb940 2024-03-16T17:57:38.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.545+0000 7f1a49200640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1a2006b3b0 0x7f1a2006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:38.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.545+0000 7f1a49200640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1a2006b3b0 0x7f1a2006d870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a34034000 tx=0x7f1a34009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:38.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.548+0000 7f1a40c00640 1 -- 172.21.15.82:0/3888728471 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1a2c054b40 con 0x7f1a440cb940 2024-03-16T17:57:38.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:38.744+0000 7f1a4bc00640 1 -- 172.21.15.82:0/3888728471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1a44158060 con 0x7f1a440cb940 2024-03-16T17:57:39.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.328+0000 7f1a40c00640 1 -- 172.21.15.82:0/3888728471 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1a44158060 con 0x7f1a440cb940 2024-03-16T17:57:39.332 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-03-16T17:57:39.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.332+0000 7f1a4bc00640 1 -- 172.21.15.82:0/3888728471 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1a2006b3b0 msgr2=0x7f1a2006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:39.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.332+0000 7f1a4bc00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1a2006b3b0 0x7f1a2006d870 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1a34034000 tx=0x7f1a34009030 comp rx=0 tx=0).stop 2024-03-16T17:57:39.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.332+0000 7f1a4bc00640 1 -- 172.21.15.82:0/3888728471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a440cb940 msgr2=0x7f1a44159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:39.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.332+0000 7f1a4bc00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a440cb940 0x7f1a44159c10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00d5e0 tx=0x7f1a2c00daa0 comp rx=0 tx=0).stop 2024-03-16T17:57:39.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.332+0000 7f1a4bc00640 1 -- 172.21.15.82:0/3888728471 shutdown_connections 2024-03-16T17:57:39.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.332+0000 7f1a4bc00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1a2006b3b0 0x7f1a2006d870 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:39.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.332+0000 7f1a4bc00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1a44156e30 0x7f1a4415bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:39.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.332+0000 7f1a4bc00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a440cb940 0x7f1a44159c10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:39.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.332+0000 7f1a4bc00640 1 --2- 172.21.15.82:0/3888728471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1a28003970 0x7f1a440cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:39.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.332+0000 7f1a4bc00640 1 -- 172.21.15.82:0/3888728471 >> 172.21.15.82:0/3888728471 conn(0x7f1a440d3db0 msgr2=0x7f1a4406b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:39.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.333+0000 7f1a4bc00640 1 -- 172.21.15.82:0/3888728471 shutdown_connections 2024-03-16T17:57:39.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.333+0000 7f1a4bc00640 1 -- 172.21.15.82:0/3888728471 wait complete. 2024-03-16T17:57:39.366 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-16T17:57:39.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.561+0000 7ff52f200640 1 Processor -- start 2024-03-16T17:57:39.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.561+0000 7ff52f200640 1 -- start start 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.562+0000 7ff52f200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5300d05a0 0x7ff5300d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.562+0000 7ff52f200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff53015e0f0 0x7ff5300d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.562+0000 7ff52f200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5300d0ec0 con 0x7ff5300cf260 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.562+0000 7ff52f200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5300b35a0 con 0x7ff5300d05a0 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.562+0000 7ff52f200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5300cf7b0 con 0x7ff53015e0f0 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.563+0000 7ff52de00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5300d05a0 0x7ff5300d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.563+0000 7ff52de00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff5300d05a0 0x7ff5300d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39490/0 (socket says 172.21.15.82:39490) 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.563+0000 7ff52de00640 1 -- 172.21.15.82:0/2074452675 learned_addr learned my addr 172.21.15.82:0/2074452675 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.563+0000 7ff52e800640 1 --1- 172.21.15.82:0/2074452675 >> v1:172.21.15.82:6789/0 conn(0x7ff5300cf260 0x7ff5300d1810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48548/0 (socket says 172.21.15.82:48548) 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.563+0000 7ff525400640 1 --2- 172.21.15.82:0/2074452675 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff53015e0f0 0x7ff5300d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.563+0000 7ff52de00640 1 -- 172.21.15.82:0/2074452675 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff53015e0f0 msgr2=0x7ff5300d4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.563+0000 7ff52de00640 1 --2- 172.21.15.82:0/2074452675 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff53015e0f0 0x7ff5300d4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.563+0000 7ff52de00640 1 -- 172.21.15.82:0/2074452675 >> v1:172.21.15.82:6789/0 conn(0x7ff5300cf260 legacy=0x7ff5300d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.563+0000 7ff52de00640 1 -- 172.21.15.82:0/2074452675 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5300cfa90 con 0x7ff5300d05a0 2024-03-16T17:57:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.563+0000 7ff52d400640 1 -- 172.21.15.82:0/2074452675 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3984068150 0 0) 0x7ff5300d0ec0 con 0x7ff5300cf260 2024-03-16T17:57:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.564+0000 7ff52de00640 1 --2- 172.21.15.82:0/2074452675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300d05a0 0x7ff5300d0980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff520009e50 tx=0x7ff52002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97347aaa8ff31b6d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52d400640 1 -- 172.21.15.82:0/2074452675 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52002cb00 con 0x7ff5300d05a0 2024-03-16T17:57:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52d400640 1 -- 172.21.15.82:0/2074452675 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52002cc60 con 0x7ff5300d05a0 2024-03-16T17:57:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52d400640 1 -- 172.21.15.82:0/2074452675 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5200343f0 con 0x7ff5300d05a0 2024-03-16T17:57:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52f200640 1 -- 172.21.15.82:0/2074452675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300d05a0 msgr2=0x7ff5300d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52f200640 1 --2- 172.21.15.82:0/2074452675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300d05a0 0x7ff5300d0980 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff520009e50 tx=0x7ff52002bf60 comp rx=0 tx=0).stop 2024-03-16T17:57:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52f200640 1 -- 172.21.15.82:0/2074452675 shutdown_connections 2024-03-16T17:57:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52f200640 1 --2- 172.21.15.82:0/2074452675 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff53015e0f0 0x7ff5300d4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52f200640 1 --2- 172.21.15.82:0/2074452675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300d05a0 0x7ff5300d0980 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52f200640 1 -- 172.21.15.82:0/2074452675 >> 172.21.15.82:0/2074452675 conn(0x7ff53009cf30 msgr2=0x7ff5300d3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52f200640 1 -- 172.21.15.82:0/2074452675 shutdown_connections 2024-03-16T17:57:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.565+0000 7ff52f200640 1 -- 172.21.15.82:0/2074452675 wait complete. 2024-03-16T17:57:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.567+0000 7ff52f200640 1 Processor -- start 2024-03-16T17:57:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.567+0000 7ff52f200640 1 -- start start 2024-03-16T17:57:39.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.568+0000 7ff52f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300cf260 0x7ff5300b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:39.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.568+0000 7ff52f200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff5300d05a0 0x7ff530151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.568+0000 7ff52f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff53015e0f0 0x7ff53009ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.568+0000 7ff52f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5301f4820 con 0x7ff5300cf260 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.568+0000 7ff52f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5300b78d0 con 0x7ff53015e0f0 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.568+0000 7ff52f200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff530151e70 con 0x7ff5300d05a0 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.568+0000 7ff52de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300cf260 0x7ff5300b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.568+0000 7ff52de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300cf260 0x7ff5300b7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39506/0 (socket says 172.21.15.82:39506) 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.568+0000 7ff52de00640 1 -- 172.21.15.82:0/677896876 learned_addr learned my addr 172.21.15.82:0/677896876 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.569+0000 7ff525400640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff5300d05a0 0x7ff530151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.569+0000 7ff52e800640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff53015e0f0 0x7ff53009ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.569+0000 7ff52de00640 1 -- 172.21.15.82:0/677896876 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff5300d05a0 msgr2=0x7ff530151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.569+0000 7ff52de00640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff5300d05a0 0x7ff530151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.569+0000 7ff52de00640 1 -- 172.21.15.82:0/677896876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff53015e0f0 msgr2=0x7ff53009ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.569+0000 7ff52de00640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff53015e0f0 0x7ff53009ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.569+0000 7ff52de00640 1 -- 172.21.15.82:0/677896876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520009b30 con 0x7ff5300cf260 2024-03-16T17:57:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.569+0000 7ff52de00640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300cf260 0x7ff5300b7d60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff520036040 tx=0x7ff520004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:39.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.569+0000 7ff526a00640 1 -- 172.21.15.82:0/677896876 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520009510 con 0x7ff5300cf260 2024-03-16T17:57:39.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.570+0000 7ff526a00640 1 -- 172.21.15.82:0/677896876 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52003e450 con 0x7ff5300cf260 2024-03-16T17:57:39.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.570+0000 7ff52f200640 1 -- 172.21.15.82:0/677896876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5300b6cc0 con 0x7ff5300cf260 2024-03-16T17:57:39.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.570+0000 7ff526a00640 1 -- 172.21.15.82:0/677896876 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52003e7c0 con 0x7ff5300cf260 2024-03-16T17:57:39.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.570+0000 7ff52f200640 1 -- 172.21.15.82:0/677896876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5300b7280 con 0x7ff5300cf260 2024-03-16T17:57:39.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.571+0000 7ff52f200640 1 -- 172.21.15.82:0/677896876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53004f250 con 0x7ff5300cf260 2024-03-16T17:57:39.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.572+0000 7ff526a00640 1 -- 172.21.15.82:0/677896876 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7ff52003e920 con 0x7ff5300cf260 2024-03-16T17:57:39.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.573+0000 7ff526a00640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff50c06b3e0 0x7ff50c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:39.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.573+0000 7ff526a00640 1 -- 172.21.15.82:0/677896876 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff5200ac8b0 con 0x7ff5300cf260 2024-03-16T17:57:39.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.573+0000 7ff525400640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff50c06b3e0 0x7ff50c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:39.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.574+0000 7ff525400640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff50c06b3e0 0x7ff50c06d8a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff518006fd0 tx=0x7ff518008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:39.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.578+0000 7ff526a00640 1 -- 172.21.15.82:0/677896876 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff520077650 con 0x7ff5300cf260 2024-03-16T17:57:39.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:39.773+0000 7ff52f200640 1 -- 172.21.15.82:0/677896876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff5300b0820 con 0x7ff5300cf260 2024-03-16T17:57:40.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.334+0000 7ff526a00640 1 -- 172.21.15.82:0/677896876 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff5200430d0 con 0x7ff5300cf260 2024-03-16T17:57:40.338 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-03-16T17:57:40.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.338+0000 7ff52f200640 1 -- 172.21.15.82:0/677896876 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff50c06b3e0 msgr2=0x7ff50c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:40.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.338+0000 7ff52f200640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff50c06b3e0 0x7ff50c06d8a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff518006fd0 tx=0x7ff518008040 comp rx=0 tx=0).stop 2024-03-16T17:57:40.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.338+0000 7ff52f200640 1 -- 172.21.15.82:0/677896876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300cf260 msgr2=0x7ff5300b7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:40.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.338+0000 7ff52f200640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300cf260 0x7ff5300b7d60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff520036040 tx=0x7ff520004a40 comp rx=0 tx=0).stop 2024-03-16T17:57:40.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.338+0000 7ff52f200640 1 -- 172.21.15.82:0/677896876 shutdown_connections 2024-03-16T17:57:40.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.338+0000 7ff52f200640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff50c06b3e0 0x7ff50c06d8a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:40.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.338+0000 7ff52f200640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff53015e0f0 0x7ff53009ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:40.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.338+0000 7ff52f200640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff5300d05a0 0x7ff530151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:40.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.338+0000 7ff52f200640 1 --2- 172.21.15.82:0/677896876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5300cf260 0x7ff5300b7d60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.338+0000 7ff52f200640 1 -- 172.21.15.82:0/677896876 >> 172.21.15.82:0/677896876 conn(0x7ff53009cf30 msgr2=0x7ff5300ce710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.339+0000 7ff52f200640 1 -- 172.21.15.82:0/677896876 shutdown_connections 2024-03-16T17:57:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.339+0000 7ff52f200640 1 -- 172.21.15.82:0/677896876 wait complete. 2024-03-16T17:57:40.376 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-16T17:57:40.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.553+0000 7fccbc200640 1 Processor -- start 2024-03-16T17:57:40.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.553+0000 7fccbc200640 1 -- start start 2024-03-16T17:57:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.555+0000 7fccbc200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fccb406e940 0x7fccb40cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.555+0000 7fccbc200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fccb40cc850 0x7fccb40cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.555+0000 7fccbc200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb415c370 con 0x7fccb4159960 2024-03-16T17:57:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.555+0000 7fccbc200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb4159d40 con 0x7fccb40cc850 2024-03-16T17:57:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.555+0000 7fccbc200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb415cff0 con 0x7fccb406e940 2024-03-16T17:57:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.555+0000 7fccb9800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fccb4159960 0x7fccb406e230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48550/0 (socket says 172.21.15.82:48550) 2024-03-16T17:57:40.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.555+0000 7fccb9800640 1 -- 172.21.15.82:0/1507718630 learned_addr learned my addr 172.21.15.82:0/1507718630 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:40.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.555+0000 7fccba200640 1 --2- 172.21.15.82:0/1507718630 >> v2:172.21.15.82:3300/0 conn(0x7fccb40cc850 0x7fccb40cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:40.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.555+0000 7fccb8e00640 1 --2- 172.21.15.82:0/1507718630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fccb406e940 0x7fccb40cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:40.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.555+0000 7fccb3e00640 1 -- 172.21.15.82:0/1507718630 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493692786 0 0) 0x7fccb415c370 con 0x7fccb4159960 2024-03-16T17:57:40.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.556+0000 7fccb3e00640 1 -- 172.21.15.82:0/1507718630 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc9c003520 con 0x7fccb4159960 2024-03-16T17:57:40.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.556+0000 7fccba200640 1 -- 172.21.15.82:0/1507718630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fccb406e940 msgr2=0x7fccb40cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:40.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.556+0000 7fccba200640 1 --2- 172.21.15.82:0/1507718630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fccb406e940 0x7fccb40cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:40.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.556+0000 7fccba200640 1 -- 172.21.15.82:0/1507718630 >> v1:172.21.15.82:6789/0 conn(0x7fccb4159960 legacy=0x7fccb406e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:40.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.556+0000 7fccba200640 1 -- 172.21.15.82:0/1507718630 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb415c650 con 0x7fccb40cc850 2024-03-16T17:57:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.557+0000 7fccba200640 1 --2- 172.21.15.82:0/1507718630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb40cc850 0x7fccb40cd540 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccac009d90 tx=0x7fccac02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2657ef2a14ce4ac5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.557+0000 7fccb3e00640 1 -- 172.21.15.82:0/1507718630 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac02cbc0 con 0x7fccb40cc850 2024-03-16T17:57:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.557+0000 7fccb3e00640 1 -- 172.21.15.82:0/1507718630 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccac02cd20 con 0x7fccb40cc850 2024-03-16T17:57:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.557+0000 7fccb3e00640 1 -- 172.21.15.82:0/1507718630 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccac034660 con 0x7fccb40cc850 2024-03-16T17:57:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.557+0000 7fccbc200640 1 -- 172.21.15.82:0/1507718630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb40cc850 msgr2=0x7fccb40cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.557+0000 7fccbc200640 1 --2- 172.21.15.82:0/1507718630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb40cc850 0x7fccb40cd540 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccac009d90 tx=0x7fccac02be10 comp rx=0 tx=0).stop 2024-03-16T17:57:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.558+0000 7fccbc200640 1 -- 172.21.15.82:0/1507718630 shutdown_connections 2024-03-16T17:57:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.558+0000 7fccbc200640 1 --2- 172.21.15.82:0/1507718630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb40cc850 0x7fccb40cd540 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.558+0000 7fccbc200640 1 --2- 172.21.15.82:0/1507718630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fccb406e940 0x7fccb40cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.558+0000 7fccbc200640 1 -- 172.21.15.82:0/1507718630 >> 172.21.15.82:0/1507718630 conn(0x7fccb40b1640 msgr2=0x7fccb40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.558+0000 7fccbc200640 1 -- 172.21.15.82:0/1507718630 shutdown_connections 2024-03-16T17:57:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.558+0000 7fccbc200640 1 -- 172.21.15.82:0/1507718630 wait complete. 2024-03-16T17:57:40.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.559+0000 7fccbc200640 1 Processor -- start 2024-03-16T17:57:40.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.559+0000 7fccbc200640 1 -- start start 2024-03-16T17:57:40.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.560+0000 7fccbc200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb406e940 0x7fccb41433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:40.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.560+0000 7fccbc200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccb40cc850 0x7fccb40cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:40.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.560+0000 7fccbc200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fccb4159960 0x7fccb40cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:40.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.560+0000 7fccbc200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb40b2730 con 0x7fccb406e940 2024-03-16T17:57:40.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.560+0000 7fccbc200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb4143910 con 0x7fccb40cc850 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.560+0000 7fccbc200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb40b29d0 con 0x7fccb4159960 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.561+0000 7fccb9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb406e940 0x7fccb41433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.561+0000 7fccb9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb406e940 0x7fccb41433d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39510/0 (socket says 172.21.15.82:39510) 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.561+0000 7fccb9800640 1 -- 172.21.15.82:0/503426110 learned_addr learned my addr 172.21.15.82:0/503426110 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.561+0000 7fccb9800640 1 -- 172.21.15.82:0/503426110 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fccb4159960 msgr2=0x7fccb40cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.561+0000 7fccb9800640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fccb4159960 0x7fccb40cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.561+0000 7fccb9800640 1 -- 172.21.15.82:0/503426110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccb40cc850 msgr2=0x7fccb40cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.561+0000 7fccb9800640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccb40cc850 0x7fccb40cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.561+0000 7fccb9800640 1 -- 172.21.15.82:0/503426110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac009a70 con 0x7fccb406e940 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.561+0000 7fccb9800640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb406e940 0x7fccb41433d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcca800c390 tx=0x7fcca800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.562+0000 7fccb1600640 1 -- 172.21.15.82:0/503426110 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca8005ce0 con 0x7fccb406e940 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.562+0000 7fccb1600640 1 -- 172.21.15.82:0/503426110 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca8005e40 con 0x7fccb406e940 2024-03-16T17:57:40.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.562+0000 7fccbc200640 1 -- 172.21.15.82:0/503426110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb40b2cb0 con 0x7fccb406e940 2024-03-16T17:57:40.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.562+0000 7fccb1600640 1 -- 172.21.15.82:0/503426110 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca80072f0 con 0x7fccb406e940 2024-03-16T17:57:40.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.562+0000 7fccbc200640 1 -- 172.21.15.82:0/503426110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb40b3270 con 0x7fccb406e940 2024-03-16T17:57:40.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.563+0000 7fccbc200640 1 -- 172.21.15.82:0/503426110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb406c980 con 0x7fccb406e940 2024-03-16T17:57:40.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.564+0000 7fccb1600640 1 -- 172.21.15.82:0/503426110 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fcca8007450 con 0x7fccb406e940 2024-03-16T17:57:40.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.564+0000 7fccb1600640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcc9406b400 0x7fcc9406d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:40.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.565+0000 7fccb1600640 1 -- 172.21.15.82:0/503426110 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fcca8089190 con 0x7fccb406e940 2024-03-16T17:57:40.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.565+0000 7fccb8e00640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcc9406b400 0x7fcc9406d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:40.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.565+0000 7fccb8e00640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcc9406b400 0x7fcc9406d8c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fccb41449d0 tx=0x7fcca0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:40.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.569+0000 7fccb1600640 1 -- 172.21.15.82:0/503426110 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fccb406c980 con 0x7fccb406e940 2024-03-16T17:57:40.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:40.795+0000 7fccbc200640 1 -- 172.21.15.82:0/503426110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fccb406c980 con 0x7fccb406e940 2024-03-16T17:57:41.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.359+0000 7fccb1600640 1 -- 172.21.15.82:0/503426110 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fccb406c980 con 0x7fccb406e940 2024-03-16T17:57:41.363 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 2 and data pool 3 2024-03-16T17:57:41.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 -- 172.21.15.82:0/503426110 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcc9406b400 msgr2=0x7fcc9406d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:41.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcc9406b400 0x7fcc9406d8c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fccb41449d0 tx=0x7fcca0008040 comp rx=0 tx=0).stop 2024-03-16T17:57:41.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 -- 172.21.15.82:0/503426110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb406e940 msgr2=0x7fccb41433d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:41.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb406e940 0x7fccb41433d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcca800c390 tx=0x7fcca800c850 comp rx=0 tx=0).stop 2024-03-16T17:57:41.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 -- 172.21.15.82:0/503426110 shutdown_connections 2024-03-16T17:57:41.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcc9406b400 0x7fcc9406d8c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fccb4159960 0x7fccb40cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccb40cc850 0x7fccb40cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 --2- 172.21.15.82:0/503426110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fccb406e940 0x7fccb41433d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 -- 172.21.15.82:0/503426110 >> 172.21.15.82:0/503426110 conn(0x7fccb40b1640 msgr2=0x7fccb40cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:41.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.363+0000 7fccbc200640 1 -- 172.21.15.82:0/503426110 shutdown_connections 2024-03-16T17:57:41.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.364+0000 7fccbc200640 1 -- 172.21.15.82:0/503426110 wait complete. 2024-03-16T17:57:41.405 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.571+0000 7f6a81600640 1 Processor -- start 2024-03-16T17:57:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.571+0000 7f6a81600640 1 -- start start 2024-03-16T17:57:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.572+0000 7f6a81600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a7c0cdf80 0x7f6a7c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.572+0000 7f6a81600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0b4b90 con 0x7f6a7c0cfe70 2024-03-16T17:57:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.572+0000 7f6a81600640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0cf500 con 0x7f6a7c1454d0 2024-03-16T17:57:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.572+0000 7f6a81600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0d0320 con 0x7f6a7c0cdf80 2024-03-16T17:57:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7b400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6a7c0cfe70 0x7f6a7c070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48552/0 (socket says 172.21.15.82:48552) 2024-03-16T17:57:41.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7b400640 1 -- 172.21.15.82:0/1736786385 learned_addr learned my addr 172.21.15.82:0/1736786385 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a73e00640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a7c0cdf80 0x7f6a7c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567025257 0 0) 0x7f6a7c0b4b90 con 0x7f6a7c0cfe70 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a60003520 con 0x7f6a7c0cfe70 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4081846111 0 0) 0x7f6a60003520 con 0x7f6a7c0cfe70 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a7c0cdf80 msgr2=0x7f6a7c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7a000640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a7c0cdf80 0x7f6a7c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 >> v1:172.21.15.90:6789/0 conn(0x7f6a7c1454d0 legacy=0x7f6a7c0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c1458b0 con 0x7f6a7c0cfe70 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f6a740019f0 con 0x7f6a7c0cfe70 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.573+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 >> v1:172.21.15.82:6789/0 conn(0x7f6a7c0cfe70 legacy=0x7f6a7c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7a000640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a7c0cdf80 0x7f6a600039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7a000640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a60004020 0x7f6a600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7a000640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a60004d00 0x7f6a60005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0d0320 con 0x7f6a60004d00 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6000bbc0 con 0x7f6a60004020 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6000bd80 con 0x7f6a7c0cdf80 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7b400640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a60004d00 0x7f6a60005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7aa00640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a7c0cdf80 0x7f6a600039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a73e00640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a60004020 0x7f6a600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7b400640 1 -- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a7c0cdf80 msgr2=0x7f6a600039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7b400640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a7c0cdf80 0x7f6a600039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7b400640 1 -- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a60004020 msgr2=0x7f6a600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7b400640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a60004020 0x7f6a600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.574+0000 7f6a7b400640 1 -- 172.21.15.82:0/1736786385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a60003520 con 0x7f6a60004d00 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.575+0000 7f6a7b400640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a60004d00 0x7f6a60005180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400a760 tx=0x7f6a600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d3a946b6d0c579 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:41.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.575+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74034630 con 0x7f6a60004d00 2024-03-16T17:57:41.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.575+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7402f070 con 0x7f6a60004d00 2024-03-16T17:57:41.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.575+0000 7f6a7a000640 1 -- 172.21.15.82:0/1736786385 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a740349f0 con 0x7f6a60004d00 2024-03-16T17:57:41.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.576+0000 7f6a81600640 1 -- 172.21.15.82:0/1736786385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a60004d00 msgr2=0x7f6a60005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:41.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.576+0000 7f6a81600640 1 --2- 172.21.15.82:0/1736786385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a60004d00 0x7f6a60005180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400a760 tx=0x7f6a600057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.576+0000 7f6a7b400640 1 -- 172.21.15.82:0/1736786385 reap_dead start 2024-03-16T17:57:41.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.576+0000 7f6a81600640 1 -- 172.21.15.82:0/1736786385 shutdown_connections 2024-03-16T17:57:41.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.576+0000 7f6a81600640 1 -- 172.21.15.82:0/1736786385 >> 172.21.15.82:0/1736786385 conn(0x7f6a7c06ae70 msgr2=0x7f6a7c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:41.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.576+0000 7f6a81600640 1 -- 172.21.15.82:0/1736786385 shutdown_connections 2024-03-16T17:57:41.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.576+0000 7f6a81600640 1 -- 172.21.15.82:0/1736786385 wait complete. 2024-03-16T17:57:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.578+0000 7f6a81600640 1 Processor -- start 2024-03-16T17:57:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.578+0000 7f6a81600640 1 -- start start 2024-03-16T17:57:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a81600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a60004d00 0x7f6a7c13af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a81600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a7c15ac80 0x7f6a7c159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a81600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c0c4e50 0x7f6a7c139df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a81600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c15a860 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a81600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1599d0 con 0x7f6a7c15ac80 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a81600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c159be0 con 0x7f6a60004d00 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a7b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c0c4e50 0x7f6a7c139df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a7b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c0c4e50 0x7f6a7c139df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39522/0 (socket says 172.21.15.82:39522) 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a7b400640 1 -- 172.21.15.82:0/3244120616 learned_addr learned my addr 172.21.15.82:0/3244120616 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a7aa00640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a60004d00 0x7f6a7c13af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a73e00640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a7c15ac80 0x7f6a7c159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a7b400640 1 -- 172.21.15.82:0/3244120616 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a60004d00 msgr2=0x7f6a7c13af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a7b400640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a60004d00 0x7f6a7c13af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a7b400640 1 -- 172.21.15.82:0/3244120616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a7c15ac80 msgr2=0x7f6a7c159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a7b400640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a7c15ac80 0x7f6a7c159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a7b400640 1 -- 172.21.15.82:0/3244120616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c159ec0 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.579+0000 7f6a7aa00640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a60004d00 0x7f6a7c13af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.580+0000 7f6a7b400640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c0c4e50 0x7f6a7c139df0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400a760 tx=0x7f6a74001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.580+0000 7f6a72a00640 1 -- 172.21.15.82:0/3244120616 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74034430 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.580+0000 7f6a72a00640 1 -- 172.21.15.82:0/3244120616 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a74034590 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.580+0000 7f6a72a00640 1 -- 172.21.15.82:0/3244120616 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74034900 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.580+0000 7f6a81600640 1 -- 172.21.15.82:0/3244120616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c15a1a0 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.580+0000 7f6a81600640 1 -- 172.21.15.82:0/3244120616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c0c5300 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.581+0000 7f6a81600640 1 -- 172.21.15.82:0/3244120616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c15a9a0 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.582+0000 7f6a72a00640 1 -- 172.21.15.82:0/3244120616 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f6a74031030 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.583+0000 7f6a72a00640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6a5806b3b0 0x7f6a5806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:41.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.583+0000 7f6a72a00640 1 -- 172.21.15.82:0/3244120616 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a740abff0 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.583+0000 7f6a7aa00640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6a5806b3b0 0x7f6a5806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:41.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.584+0000 7f6a7aa00640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6a5806b3b0 0x7f6a5806d870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c000c00 tx=0x7f6a6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:41.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.588+0000 7f6a72a00640 1 -- 172.21.15.82:0/3244120616 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6a7c15a9a0 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.783+0000 7f6a81600640 1 -- 172.21.15.82:0/3244120616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a7c15a9a0 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.786+0000 7f6a72a00640 1 -- 172.21.15.82:0/3244120616 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f6a7c15a9a0 con 0x7f6a7c0c4e50 2024-03-16T17:57:41.790 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:41.790 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":20,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:40.797338+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 -- 172.21.15.82:0/3244120616 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6a5806b3b0 msgr2=0x7f6a5806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6a5806b3b0 0x7f6a5806d870 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c000c00 tx=0x7f6a6c009040 comp rx=0 tx=0).stop 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 -- 172.21.15.82:0/3244120616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c0c4e50 msgr2=0x7f6a7c139df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c0c4e50 0x7f6a7c139df0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400a760 tx=0x7f6a74001cb0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 -- 172.21.15.82:0/3244120616 shutdown_connections 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6a5806b3b0 0x7f6a5806d870 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7c0c4e50 0x7f6a7c139df0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a7c15ac80 0x7f6a7c159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 --2- 172.21.15.82:0/3244120616 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6a60004d00 0x7f6a7c13af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 -- 172.21.15.82:0/3244120616 >> 172.21.15.82:0/3244120616 conn(0x7f6a7c06ae70 msgr2=0x7f6a7c1446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 -- 172.21.15.82:0/3244120616 shutdown_connections 2024-03-16T17:57:41.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:41.788+0000 7f6a81600640 1 -- 172.21.15.82:0/3244120616 wait complete. 2024-03-16T17:57:41.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:42.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.001+0000 7f0e88c00640 1 Processor -- start 2024-03-16T17:57:42.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.001+0000 7f0e88c00640 1 -- start start 2024-03-16T17:57:42.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.003+0000 7f0e88c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0e840d0e10 0x7f0e840d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.003+0000 7f0e88c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8415e5b0 con 0x7f0e8415e1d0 2024-03-16T17:57:42.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.003+0000 7f0e88c00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e840d28c0 con 0x7f0e840d2410 2024-03-16T17:57:42.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.003+0000 7f0e88c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e840d2b60 con 0x7f0e840d0e10 2024-03-16T17:57:42.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.003+0000 7f0e81400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0e840d0e10 0x7f0e840d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:42.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.003+0000 7f0e81e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0e8415e1d0 0x7f0e840d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48560/0 (socket says 172.21.15.82:48560) 2024-03-16T17:57:42.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.003+0000 7f0e81e00640 1 -- 172.21.15.82:0/1997301927 learned_addr learned my addr 172.21.15.82:0/1997301927 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:42.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.004+0000 7f0e81400640 1 -- 172.21.15.82:0/1997301927 >> v1:172.21.15.90:6789/0 conn(0x7f0e840d2410 legacy=0x7f0e840b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:42.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.004+0000 7f0e81400640 1 -- 172.21.15.82:0/1997301927 >> v1:172.21.15.82:6789/0 conn(0x7f0e8415e1d0 legacy=0x7f0e840d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:42.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.004+0000 7f0e81400640 1 -- 172.21.15.82:0/1997301927 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e840d2e40 con 0x7f0e840d0e10 2024-03-16T17:57:42.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.005+0000 7f0e81400640 1 --2- 172.21.15.82:0/1997301927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e840d0e10 0x7f0e840d1310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009dc0 tx=0x7f0e6c02bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f799ea0ed5472573 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:42.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.005+0000 7f0e80a00640 1 -- 172.21.15.82:0/1997301927 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c02ccd0 con 0x7f0e840d0e10 2024-03-16T17:57:42.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.005+0000 7f0e80a00640 1 -- 172.21.15.82:0/1997301927 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c02ce30 con 0x7f0e840d0e10 2024-03-16T17:57:42.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.005+0000 7f0e80a00640 1 -- 172.21.15.82:0/1997301927 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c034660 con 0x7f0e840d0e10 2024-03-16T17:57:42.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.006+0000 7f0e88c00640 1 -- 172.21.15.82:0/1997301927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e840d0e10 msgr2=0x7f0e840d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:42.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.006+0000 7f0e88c00640 1 --2- 172.21.15.82:0/1997301927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e840d0e10 0x7f0e840d1310 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009dc0 tx=0x7f0e6c02bf20 comp rx=0 tx=0).stop 2024-03-16T17:57:42.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.006+0000 7f0e88c00640 1 -- 172.21.15.82:0/1997301927 shutdown_connections 2024-03-16T17:57:42.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.006+0000 7f0e88c00640 1 --2- 172.21.15.82:0/1997301927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e840d0e10 0x7f0e840d1310 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.006+0000 7f0e88c00640 1 -- 172.21.15.82:0/1997301927 >> 172.21.15.82:0/1997301927 conn(0x7f0e840d3e30 msgr2=0x7f0e840d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:42.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.006+0000 7f0e88c00640 1 -- 172.21.15.82:0/1997301927 shutdown_connections 2024-03-16T17:57:42.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.006+0000 7f0e88c00640 1 -- 172.21.15.82:0/1997301927 wait complete. 2024-03-16T17:57:42.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.008+0000 7f0e88c00640 1 Processor -- start 2024-03-16T17:57:42.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.008+0000 7f0e88c00640 1 -- start start 2024-03-16T17:57:42.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.009+0000 7f0e88c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e840d0e10 0x7f0e840cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.009+0000 7f0e88c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e840d2410 0x7f0e84151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.009+0000 7f0e88c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e8415e1d0 0x7f0e8415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.009+0000 7f0e88c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840cd8a0 con 0x7f0e8415e1d0 2024-03-16T17:57:42.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.009+0000 7f0e88c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840ccf00 con 0x7f0e840d2410 2024-03-16T17:57:42.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.009+0000 7f0e88c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e840cd1e0 con 0x7f0e840d0e10 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e82800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e8415e1d0 0x7f0e8415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e82800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e8415e1d0 0x7f0e8415bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39552/0 (socket says 172.21.15.82:39552) 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e82800640 1 -- 172.21.15.82:0/2569295432 learned_addr learned my addr 172.21.15.82:0/2569295432 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e81e00640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e840d0e10 0x7f0e840cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e81400640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e840d2410 0x7f0e84151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e82800640 1 -- 172.21.15.82:0/2569295432 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e840d0e10 msgr2=0x7f0e840cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e82800640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e840d0e10 0x7f0e840cdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e82800640 1 -- 172.21.15.82:0/2569295432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e840d2410 msgr2=0x7f0e84151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e82800640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e840d2410 0x7f0e84151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e82800640 1 -- 172.21.15.82:0/2569295432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c009a70 con 0x7f0e8415e1d0 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.010+0000 7f0e82800640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e8415e1d0 0x7f0e8415bcc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c410 tx=0x7f0e7c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.011+0000 7f0e7a000640 1 -- 172.21.15.82:0/2569295432 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c011790 con 0x7f0e8415e1d0 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.011+0000 7f0e7a000640 1 -- 172.21.15.82:0/2569295432 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c0118f0 con 0x7f0e8415e1d0 2024-03-16T17:57:42.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.011+0000 7f0e7a000640 1 -- 172.21.15.82:0/2569295432 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c011ca0 con 0x7f0e8415e1d0 2024-03-16T17:57:42.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.011+0000 7f0e88c00640 1 -- 172.21.15.82:0/2569295432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e840cd4c0 con 0x7f0e8415e1d0 2024-03-16T17:57:42.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.011+0000 7f0e88c00640 1 -- 172.21.15.82:0/2569295432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e840c8d40 con 0x7f0e8415e1d0 2024-03-16T17:57:42.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.012+0000 7f0e88c00640 1 -- 172.21.15.82:0/2569295432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e840cd9e0 con 0x7f0e8415e1d0 2024-03-16T17:57:42.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.013+0000 7f0e7a000640 1 -- 172.21.15.82:0/2569295432 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f0e7c01c460 con 0x7f0e8415e1d0 2024-03-16T17:57:42.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.014+0000 7f0e7a000640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 0x7f0e6006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.014+0000 7f0e7a000640 1 -- 172.21.15.82:0/2569295432 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e7c0898f0 con 0x7f0e8415e1d0 2024-03-16T17:57:42.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.014+0000 7f0e81e00640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 0x7f0e6006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:42.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.015+0000 7f0e81e00640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 0x7f0e6006d870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e74000f60 tx=0x7f0e74009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:42.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.018+0000 7f0e7a000640 1 -- 172.21.15.82:0/2569295432 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0e840cd9e0 con 0x7f0e8415e1d0 2024-03-16T17:57:42.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.218+0000 7f0e88c00640 1 -- 172.21.15.82:0/2569295432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e840cd9e0 con 0x7f0e8415e1d0 2024-03-16T17:57:42.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.220+0000 7f0e7a000640 1 -- 172.21.15.82:0/2569295432 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f0e840cd9e0 con 0x7f0e8415e1d0 2024-03-16T17:57:42.224 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:42.224 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":20,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:40.797338+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:42.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.222+0000 7f0e88c00640 1 -- 172.21.15.82:0/2569295432 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 msgr2=0x7f0e6006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:42.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.222+0000 7f0e88c00640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 0x7f0e6006d870 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0e74000f60 tx=0x7f0e74009040 comp rx=0 tx=0).stop 2024-03-16T17:57:42.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.222+0000 7f0e88c00640 1 -- 172.21.15.82:0/2569295432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e8415e1d0 msgr2=0x7f0e8415bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:42.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.222+0000 7f0e88c00640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e8415e1d0 0x7f0e8415bcc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c410 tx=0x7f0e7c00c8d0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.223+0000 7f0e88c00640 1 -- 172.21.15.82:0/2569295432 shutdown_connections 2024-03-16T17:57:42.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.223+0000 7f0e88c00640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 0x7f0e6006d870 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.223+0000 7f0e88c00640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e8415e1d0 0x7f0e8415bcc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.223+0000 7f0e88c00640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e840d2410 0x7f0e84151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.223+0000 7f0e88c00640 1 --2- 172.21.15.82:0/2569295432 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e840d0e10 0x7f0e840cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.223+0000 7f0e88c00640 1 -- 172.21.15.82:0/2569295432 >> 172.21.15.82:0/2569295432 conn(0x7f0e840d3e30 msgr2=0x7f0e8406b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:42.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.223+0000 7f0e88c00640 1 -- 172.21.15.82:0/2569295432 shutdown_connections 2024-03-16T17:57:42.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.223+0000 7f0e88c00640 1 -- 172.21.15.82:0/2569295432 wait complete. 2024-03-16T17:57:42.245 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-16T17:57:42.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.454+0000 7f7947000640 1 Processor -- start 2024-03-16T17:57:42.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.454+0000 7f7947000640 1 -- start start 2024-03-16T17:57:42.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7947000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f79400b4770 0x7f794015e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7947000640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f79400aed30 0x7f79400b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7947000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79400b36b0 0x7f794006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7947000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79400b4b50 con 0x7f79400b4770 2024-03-16T17:57:42.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7947000640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79400b29b0 con 0x7f79400aed30 2024-03-16T17:57:42.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7947000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794015ecf0 con 0x7f79400b36b0 2024-03-16T17:57:42.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7945c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f79400b4770 0x7f794015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:42.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7945c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f79400b4770 0x7f794015e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39556/0 (socket says 172.21.15.82:39556) 2024-03-16T17:57:42.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7945c00640 1 -- 172.21.15.82:0/2221422691 learned_addr learned my addr 172.21.15.82:0/2221422691 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:42.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7945200640 1 --2- 172.21.15.82:0/2221422691 >> v2:172.21.15.90:3300/0 conn(0x7f79400aed30 0x7f79400b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:42.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7945c00640 1 -- 172.21.15.82:0/2221422691 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79400b36b0 msgr2=0x7f794006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:42.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7945c00640 1 --2- 172.21.15.82:0/2221422691 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79400b36b0 0x7f794006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7945c00640 1 -- 172.21.15.82:0/2221422691 >> v2:172.21.15.90:3300/0 conn(0x7f79400aed30 msgr2=0x7f79400b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:42.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7945c00640 1 --2- 172.21.15.82:0/2221422691 >> v2:172.21.15.90:3300/0 conn(0x7f79400aed30 0x7f79400b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7945c00640 1 -- 172.21.15.82:0/2221422691 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794015efd0 con 0x7f79400b4770 2024-03-16T17:57:42.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.456+0000 7f7945200640 1 --2- 172.21.15.82:0/2221422691 >> v2:172.21.15.90:3300/0 conn(0x7f79400aed30 0x7f79400b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:42.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.457+0000 7f7945c00640 1 --2- 172.21.15.82:0/2221422691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400b4770 0x7f794015e430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f79400af1c0 tx=0x7f793402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ef95c7daeae63 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:42.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.458+0000 7f793fe00640 1 -- 172.21.15.82:0/2221422691 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934031870 con 0x7f79400b4770 2024-03-16T17:57:42.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.458+0000 7f793fe00640 1 -- 172.21.15.82:0/2221422691 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79340319d0 con 0x7f79400b4770 2024-03-16T17:57:42.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.458+0000 7f793fe00640 1 -- 172.21.15.82:0/2221422691 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934031d60 con 0x7f79400b4770 2024-03-16T17:57:42.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.458+0000 7f7947000640 1 -- 172.21.15.82:0/2221422691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400b4770 msgr2=0x7f794015e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:42.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.458+0000 7f7947000640 1 --2- 172.21.15.82:0/2221422691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400b4770 0x7f794015e430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f79400af1c0 tx=0x7f793402b910 comp rx=0 tx=0).stop 2024-03-16T17:57:42.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.459+0000 7f7947000640 1 -- 172.21.15.82:0/2221422691 shutdown_connections 2024-03-16T17:57:42.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.459+0000 7f7947000640 1 --2- 172.21.15.82:0/2221422691 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79400b36b0 0x7f794006d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.459+0000 7f7947000640 1 --2- 172.21.15.82:0/2221422691 >> v2:172.21.15.90:3300/0 conn(0x7f79400aed30 0x7f79400b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.459+0000 7f7947000640 1 --2- 172.21.15.82:0/2221422691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400b4770 0x7f794015e430 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.459+0000 7f7947000640 1 -- 172.21.15.82:0/2221422691 >> 172.21.15.82:0/2221422691 conn(0x7f794006ae70 msgr2=0x7f794006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:42.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.459+0000 7f7947000640 1 -- 172.21.15.82:0/2221422691 shutdown_connections 2024-03-16T17:57:42.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.459+0000 7f7947000640 1 -- 172.21.15.82:0/2221422691 wait complete. 2024-03-16T17:57:42.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.460+0000 7f7947000640 1 Processor -- start 2024-03-16T17:57:42.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.460+0000 7f7947000640 1 -- start start 2024-03-16T17:57:42.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.461+0000 7f7947000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400aed30 0x7f7940146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.461+0000 7f7947000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79400b36b0 0x7f79400cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.461+0000 7f7947000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79400b4770 0x7f79400ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.461+0000 7f7947000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79401f4000 con 0x7f79400aed30 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.461+0000 7f7947000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7940147230 con 0x7f79400b4770 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.461+0000 7f7947000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7940147760 con 0x7f79400b36b0 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.461+0000 7f7945c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400aed30 0x7f7940146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f7945c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400aed30 0x7f7940146cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39570/0 (socket says 172.21.15.82:39570) 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f7945c00640 1 -- 172.21.15.82:0/4026908689 learned_addr learned my addr 172.21.15.82:0/4026908689 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f7945c00640 1 -- 172.21.15.82:0/4026908689 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79400b36b0 msgr2=0x7f79400cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f7945c00640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79400b36b0 0x7f79400cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f7945c00640 1 -- 172.21.15.82:0/4026908689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79400b4770 msgr2=0x7f79400ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f7945c00640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79400b4770 0x7f79400ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f7945c00640 1 -- 172.21.15.82:0/4026908689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340094d0 con 0x7f79400aed30 2024-03-16T17:57:42.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f7945c00640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400aed30 0x7f7940146cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7934000c00 tx=0x7f79340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:42.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f793d600640 1 -- 172.21.15.82:0/4026908689 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934031840 con 0x7f79400aed30 2024-03-16T17:57:42.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f793d600640 1 -- 172.21.15.82:0/4026908689 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79340443f0 con 0x7f79400aed30 2024-03-16T17:57:42.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f793d600640 1 -- 172.21.15.82:0/4026908689 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79340447a0 con 0x7f79400aed30 2024-03-16T17:57:42.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.462+0000 7f7947000640 1 -- 172.21.15.82:0/4026908689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79401596d0 con 0x7f79400aed30 2024-03-16T17:57:42.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.463+0000 7f7947000640 1 -- 172.21.15.82:0/4026908689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7940159c90 con 0x7f79400aed30 2024-03-16T17:57:42.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.464+0000 7f7947000640 1 -- 172.21.15.82:0/4026908689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79400b27d0 con 0x7f79400aed30 2024-03-16T17:57:42.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.465+0000 7f793d600640 1 -- 172.21.15.82:0/4026908689 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f793404d450 con 0x7f79400aed30 2024-03-16T17:57:42.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.466+0000 7f793d600640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f792406b3b0 0x7f792406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:42.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.466+0000 7f793d600640 1 -- 172.21.15.82:0/4026908689 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f793402d040 con 0x7f79400aed30 2024-03-16T17:57:42.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.466+0000 7f7945200640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f792406b3b0 0x7f792406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:42.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.467+0000 7f7945200640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f792406b3b0 0x7f792406d870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7930006fd0 tx=0x7f7930008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:42.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.470+0000 7f793d600640 1 -- 172.21.15.82:0/4026908689 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7934077110 con 0x7f79400aed30 2024-03-16T17:57:42.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:42.702+0000 7f7947000640 1 -- 172.21.15.82:0/4026908689 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f794004f250 con 0x7f79400aed30 2024-03-16T17:57:43.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.398+0000 7f793d600640 1 -- 172.21.15.82:0/4026908689 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f793407a7d0 con 0x7f79400aed30 2024-03-16T17:57:43.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.402+0000 7f7947000640 1 -- 172.21.15.82:0/4026908689 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f792406b3b0 msgr2=0x7f792406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:43.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.402+0000 7f7947000640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f792406b3b0 0x7f792406d870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7930006fd0 tx=0x7f7930008040 comp rx=0 tx=0).stop 2024-03-16T17:57:43.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.402+0000 7f7947000640 1 -- 172.21.15.82:0/4026908689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400aed30 msgr2=0x7f7940146cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:43.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.402+0000 7f7947000640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400aed30 0x7f7940146cf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7934000c00 tx=0x7f79340046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:43.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.402+0000 7f7947000640 1 -- 172.21.15.82:0/4026908689 shutdown_connections 2024-03-16T17:57:43.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.402+0000 7f7947000640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f792406b3b0 0x7f792406d870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:43.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.402+0000 7f7947000640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79400b4770 0x7f79400ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:43.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.402+0000 7f7947000640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f79400b36b0 0x7f79400cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:43.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.402+0000 7f7947000640 1 --2- 172.21.15.82:0/4026908689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f79400aed30 0x7f7940146cf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:43.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.402+0000 7f7947000640 1 -- 172.21.15.82:0/4026908689 >> 172.21.15.82:0/4026908689 conn(0x7f794006ae70 msgr2=0x7f79400ae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:43.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.403+0000 7f7947000640 1 -- 172.21.15.82:0/4026908689 shutdown_connections 2024-03-16T17:57:43.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.403+0000 7f7947000640 1 -- 172.21.15.82:0/4026908689 wait complete. 2024-03-16T17:57:43.427 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-16T17:57:43.428 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-16T17:57:43.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.616+0000 7f6c5b600640 1 Processor -- start 2024-03-16T17:57:43.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.616+0000 7f6c5b600640 1 -- start start 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.618+0000 7f6c5b600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c54147060 0x7f6c5406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.618+0000 7f6c5b600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c5406ea70 0x7f6c540cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.618+0000 7f6c5b600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c54146da0 con 0x7f6c541469c0 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.618+0000 7f6c5b600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5406d490 con 0x7f6c5406ea70 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.618+0000 7f6c5b600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5406d770 con 0x7f6c54147060 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.618+0000 7f6c58c00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6c541469c0 0x7f6c54162360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48562/0 (socket says 172.21.15.82:48562) 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.618+0000 7f6c58c00640 1 -- 172.21.15.82:0/3062369986 learned_addr learned my addr 172.21.15.82:0/3062369986 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.618+0000 7f6c59600640 1 --2- 172.21.15.82:0/3062369986 >> v2:172.21.15.82:3300/0 conn(0x7f6c5406ea70 0x7f6c540cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.619+0000 7f6c4be00640 1 --2- 172.21.15.82:0/3062369986 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c54147060 0x7f6c5406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.619+0000 7f6c53e00640 1 -- 172.21.15.82:0/3062369986 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547271638 0 0) 0x7f6c54146da0 con 0x7f6c541469c0 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.619+0000 7f6c53e00640 1 -- 172.21.15.82:0/3062369986 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c38003520 con 0x7f6c541469c0 2024-03-16T17:57:43.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.619+0000 7f6c59600640 1 -- 172.21.15.82:0/3062369986 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c54147060 msgr2=0x7f6c5406e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.619+0000 7f6c59600640 1 --2- 172.21.15.82:0/3062369986 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c54147060 0x7f6c5406e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.619+0000 7f6c59600640 1 -- 172.21.15.82:0/3062369986 >> v1:172.21.15.82:6789/0 conn(0x7f6c541469c0 legacy=0x7f6c54162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.619+0000 7f6c59600640 1 -- 172.21.15.82:0/3062369986 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5406da50 con 0x7f6c5406ea70 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.619+0000 7f6c4be00640 1 --2- 172.21.15.82:0/3062369986 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c54147060 0x7f6c5406e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.620+0000 7f6c59600640 1 --2- 172.21.15.82:0/3062369986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c5406ea70 0x7f6c540cddc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009dc0 tx=0x7f6c4c02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c3b896ea0a0b87 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.621+0000 7f6c53e00640 1 -- 172.21.15.82:0/3062369986 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c02cc40 con 0x7f6c5406ea70 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.621+0000 7f6c53e00640 1 -- 172.21.15.82:0/3062369986 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c02cda0 con 0x7f6c5406ea70 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.621+0000 7f6c53e00640 1 -- 172.21.15.82:0/3062369986 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c034660 con 0x7f6c5406ea70 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.621+0000 7f6c5b600640 1 -- 172.21.15.82:0/3062369986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c5406ea70 msgr2=0x7f6c540cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.621+0000 7f6c5b600640 1 --2- 172.21.15.82:0/3062369986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c5406ea70 0x7f6c540cddc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009dc0 tx=0x7f6c4c02be90 comp rx=0 tx=0).stop 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.621+0000 7f6c5b600640 1 -- 172.21.15.82:0/3062369986 shutdown_connections 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.621+0000 7f6c5b600640 1 --2- 172.21.15.82:0/3062369986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c5406ea70 0x7f6c540cddc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.621+0000 7f6c5b600640 1 --2- 172.21.15.82:0/3062369986 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c54147060 0x7f6c5406e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:43.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.621+0000 7f6c5b600640 1 -- 172.21.15.82:0/3062369986 >> 172.21.15.82:0/3062369986 conn(0x7f6c540b1640 msgr2=0x7f6c540b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:43.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.621+0000 7f6c5b600640 1 -- 172.21.15.82:0/3062369986 shutdown_connections 2024-03-16T17:57:43.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.622+0000 7f6c5b600640 1 -- 172.21.15.82:0/3062369986 wait complete. 2024-03-16T17:57:43.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.623+0000 7f6c5b600640 1 Processor -- start 2024-03-16T17:57:43.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.623+0000 7f6c5b600640 1 -- start start 2024-03-16T17:57:43.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.624+0000 7f6c5b600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c5406ea70 0x7f6c541f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:43.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.624+0000 7f6c5b600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c541469c0 0x7f6c54070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:43.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.624+0000 7f6c5b600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c54147060 0x7f6c54070d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:43.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.624+0000 7f6c5b600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541419b0 con 0x7f6c54147060 2024-03-16T17:57:43.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.624+0000 7f6c5b600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540b3650 con 0x7f6c541469c0 2024-03-16T17:57:43.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.624+0000 7f6c5b600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540b41d0 con 0x7f6c5406ea70 2024-03-16T17:57:43.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c58c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c5406ea70 0x7f6c541f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:43.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c59600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c54147060 0x7f6c54070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:43.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c4be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c541469c0 0x7f6c54070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:43.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c59600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c54147060 0x7f6c54070d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39582/0 (socket says 172.21.15.82:39582) 2024-03-16T17:57:43.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c58c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c5406ea70 0x7f6c541f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:34264/0 (socket says 172.21.15.82:34264) 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c59600640 1 -- 172.21.15.82:0/3093919961 learned_addr learned my addr 172.21.15.82:0/3093919961 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c59600640 1 -- 172.21.15.82:0/3093919961 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c5406ea70 msgr2=0x7f6c541f37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c59600640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c5406ea70 0x7f6c541f37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c59600640 1 -- 172.21.15.82:0/3093919961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c541469c0 msgr2=0x7f6c54070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c59600640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c541469c0 0x7f6c54070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.625+0000 7f6c59600640 1 -- 172.21.15.82:0/3093919961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c009a70 con 0x7f6c54147060 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.626+0000 7f6c59600640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c54147060 0x7f6c54070d80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c02c4a0 tx=0x7f6c4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.626+0000 7f6c51600640 1 -- 172.21.15.82:0/3093919961 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c040650 con 0x7f6c54147060 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.626+0000 7f6c51600640 1 -- 172.21.15.82:0/3093919961 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c0407d0 con 0x7f6c54147060 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.626+0000 7f6c51600640 1 -- 172.21.15.82:0/3093919961 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c040b80 con 0x7f6c54147060 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.626+0000 7f6c5b600640 1 -- 172.21.15.82:0/3093919961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c540b38a0 con 0x7f6c54147060 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.626+0000 7f6c5b600640 1 -- 172.21.15.82:0/3093919961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c540b3e60 con 0x7f6c54147060 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.627+0000 7f6c5b600640 1 -- 172.21.15.82:0/3093919961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c540b4340 con 0x7f6c54147060 2024-03-16T17:57:43.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.628+0000 7f6c51600640 1 -- 172.21.15.82:0/3093919961 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f6c4c040ce0 con 0x7f6c54147060 2024-03-16T17:57:43.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.629+0000 7f6c51600640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6c3006b3e0 0x7f6c3006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:43.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.629+0000 7f6c51600640 1 -- 172.21.15.82:0/3093919961 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c4c0ac7a0 con 0x7f6c54147060 2024-03-16T17:57:43.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.629+0000 7f6c58c00640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6c3006b3e0 0x7f6c3006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:43.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.630+0000 7f6c58c00640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6c3006b3e0 0x7f6c3006d8a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c44005880 tx=0x7f6c44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:43.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.633+0000 7f6c51600640 1 -- 172.21.15.82:0/3093919961 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c4c077540 con 0x7f6c54147060 2024-03-16T17:57:43.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:43.863+0000 7f6c5b600640 1 -- 172.21.15.82:0/3093919961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6c54067f30 con 0x7f6c54147060 2024-03-16T17:57:44.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.419+0000 7f6c51600640 1 -- 172.21.15.82:0/3093919961 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6c4c009450 con 0x7f6c54147060 2024-03-16T17:57:44.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.423+0000 7f6c5b600640 1 -- 172.21.15.82:0/3093919961 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6c3006b3e0 msgr2=0x7f6c3006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:44.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.423+0000 7f6c5b600640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6c3006b3e0 0x7f6c3006d8a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6c44005880 tx=0x7f6c44009040 comp rx=0 tx=0).stop 2024-03-16T17:57:44.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.423+0000 7f6c5b600640 1 -- 172.21.15.82:0/3093919961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c54147060 msgr2=0x7f6c54070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:44.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.423+0000 7f6c5b600640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c54147060 0x7f6c54070d80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c02c4a0 tx=0x7f6c4c004a40 comp rx=0 tx=0).stop 2024-03-16T17:57:44.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.423+0000 7f6c5b600640 1 -- 172.21.15.82:0/3093919961 shutdown_connections 2024-03-16T17:57:44.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.423+0000 7f6c5b600640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6c3006b3e0 0x7f6c3006d8a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.423+0000 7f6c5b600640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c54147060 0x7f6c54070d80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.423+0000 7f6c5b600640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6c541469c0 0x7f6c54070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.423+0000 7f6c5b600640 1 --2- 172.21.15.82:0/3093919961 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6c5406ea70 0x7f6c541f37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.424+0000 7f6c5b600640 1 -- 172.21.15.82:0/3093919961 >> 172.21.15.82:0/3093919961 conn(0x7f6c540b1640 msgr2=0x7f6c540b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:44.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.424+0000 7f6c5b600640 1 -- 172.21.15.82:0/3093919961 shutdown_connections 2024-03-16T17:57:44.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.424+0000 7f6c5b600640 1 -- 172.21.15.82:0/3093919961 wait complete. 2024-03-16T17:57:44.446 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-16T17:57:44.446 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:57:44.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.642+0000 7f7062200640 1 Processor -- start 2024-03-16T17:57:44.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.643+0000 7f7062200640 1 -- start start 2024-03-16T17:57:44.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.644+0000 7f7062200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f705c0d09a0 0x7f705c168bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.644+0000 7f7062200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c0ae1f0 con 0x7f705c06eff0 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.644+0000 7f7062200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c0d0fc0 con 0x7f705c15e040 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.644+0000 7f7062200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c0d0dd0 con 0x7f705c0d09a0 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.644+0000 7f705be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f705c0d09a0 0x7f705c168bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.644+0000 7f7052a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f705c06eff0 0x7f705c0d0290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48570/0 (socket says 172.21.15.82:48570) 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.644+0000 7f7052a00640 1 -- 172.21.15.82:0/3948608274 learned_addr learned my addr 172.21.15.82:0/3948608274 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.644+0000 7f705be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f705c0d09a0 0x7f705c168bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39596/0 (socket says 172.21.15.82:39596) 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.645+0000 7f705be00640 1 -- 172.21.15.82:0/3948608274 >> v1:172.21.15.90:6789/0 conn(0x7f705c15e040 legacy=0x7f705c06e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.645+0000 7f705be00640 1 -- 172.21.15.82:0/3948608274 >> v1:172.21.15.82:6789/0 conn(0x7f705c06eff0 legacy=0x7f705c0d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.645+0000 7f705be00640 1 -- 172.21.15.82:0/3948608274 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c072620 con 0x7f705c0d09a0 2024-03-16T17:57:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.645+0000 7f705aa00640 1 -- 172.21.15.82:0/3948608274 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428189734 0 0) 0x7f705c0ae1f0 con 0x7f705c06eff0 2024-03-16T17:57:44.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.646+0000 7f705be00640 1 --2- 172.21.15.82:0/3948608274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c0d09a0 0x7f705c168bb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7054009d90 tx=0x7f705402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e25d18371283e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:44.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.646+0000 7f705aa00640 1 -- 172.21.15.82:0/3948608274 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705402cbd0 con 0x7f705c0d09a0 2024-03-16T17:57:44.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.646+0000 7f705aa00640 1 -- 172.21.15.82:0/3948608274 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705402cd30 con 0x7f705c0d09a0 2024-03-16T17:57:44.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.646+0000 7f705aa00640 1 -- 172.21.15.82:0/3948608274 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705402cea0 con 0x7f705c0d09a0 2024-03-16T17:57:44.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.647+0000 7f7062200640 1 -- 172.21.15.82:0/3948608274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c0d09a0 msgr2=0x7f705c168bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:44.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.647+0000 7f7062200640 1 --2- 172.21.15.82:0/3948608274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c0d09a0 0x7f705c168bb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7054009d90 tx=0x7f705402be20 comp rx=0 tx=0).stop 2024-03-16T17:57:44.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.647+0000 7f7062200640 1 -- 172.21.15.82:0/3948608274 shutdown_connections 2024-03-16T17:57:44.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.647+0000 7f7062200640 1 --2- 172.21.15.82:0/3948608274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c0d09a0 0x7f705c168bb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.647+0000 7f7062200640 1 -- 172.21.15.82:0/3948608274 >> 172.21.15.82:0/3948608274 conn(0x7f705c0b5060 msgr2=0x7f705c0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:44.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.647+0000 7f7062200640 1 -- 172.21.15.82:0/3948608274 shutdown_connections 2024-03-16T17:57:44.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.647+0000 7f7062200640 1 -- 172.21.15.82:0/3948608274 wait complete. 2024-03-16T17:57:44.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.649+0000 7f7062200640 1 Processor -- start 2024-03-16T17:57:44.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.650+0000 7f7062200640 1 -- start start 2024-03-16T17:57:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f7062200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06eff0 0x7f705c160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f7062200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f705c0d09a0 0x7f705c0b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f7062200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f705c15e040 0x7f705c150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:44.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f7062200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c161520 con 0x7f705c06eff0 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f7062200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0c9d50 con 0x7f705c15e040 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f7062200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0ca030 con 0x7f705c0d09a0 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f705b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06eff0 0x7f705c160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f705b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06eff0 0x7f705c160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39608/0 (socket says 172.21.15.82:39608) 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f705b400640 1 -- 172.21.15.82:0/1784404469 learned_addr learned my addr 172.21.15.82:0/1784404469 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f7052a00640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f705c0d09a0 0x7f705c0b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.651+0000 7f705be00640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f705c15e040 0x7f705c150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f705b400640 1 -- 172.21.15.82:0/1784404469 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f705c0d09a0 msgr2=0x7f705c0b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f705b400640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f705c0d09a0 0x7f705c0b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f705b400640 1 -- 172.21.15.82:0/1784404469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f705c15e040 msgr2=0x7f705c150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f705b400640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f705c15e040 0x7f705c150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f705b400640 1 -- 172.21.15.82:0/1784404469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054009a70 con 0x7f705c06eff0 2024-03-16T17:57:44.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f705be00640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f705c15e040 0x7f705c150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f705b400640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06eff0 0x7f705c160fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f704c00b3f0 tx=0x7f704c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f7053e00640 1 -- 172.21.15.82:0/1784404469 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c011830 con 0x7f705c06eff0 2024-03-16T17:57:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f7053e00640 1 -- 172.21.15.82:0/1784404469 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c011990 con 0x7f705c06eff0 2024-03-16T17:57:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f7053e00640 1 -- 172.21.15.82:0/1784404469 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c011d40 con 0x7f705c06eff0 2024-03-16T17:57:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f7062200640 1 -- 172.21.15.82:0/1784404469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c0ca310 con 0x7f705c06eff0 2024-03-16T17:57:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.652+0000 7f7062200640 1 -- 172.21.15.82:0/1784404469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c0ca8d0 con 0x7f705c06eff0 2024-03-16T17:57:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.653+0000 7f7062200640 1 -- 172.21.15.82:0/1784404469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c067f30 con 0x7f705c06eff0 2024-03-16T17:57:44.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.654+0000 7f7053e00640 1 -- 172.21.15.82:0/1784404469 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f704c01a400 con 0x7f705c06eff0 2024-03-16T17:57:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.656+0000 7f7053e00640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703806b3b0 0x7f703806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:44.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.656+0000 7f7053e00640 1 -- 172.21.15.82:0/1784404469 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f704c089b70 con 0x7f705c06eff0 2024-03-16T17:57:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.656+0000 7f7052a00640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703806b3b0 0x7f703806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:44.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.657+0000 7f7052a00640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703806b3b0 0x7f703806d870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7044007fd0 tx=0x7f7044009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:44.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.660+0000 7f7053e00640 1 -- 172.21.15.82:0/1784404469 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f704c0546a0 con 0x7f705c06eff0 2024-03-16T17:57:44.678 INFO:tasks.ceph.mds.b.smithi090.stderr:2024-03-16T17:57:44.674+0000 7f9bb4800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T17:57:44.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.877+0000 7f7062200640 1 -- 172.21.15.82:0/1784404469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f705c0ad7b0 con 0x7f705c06eff0 2024-03-16T17:57:44.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.878+0000 7f7053e00640 1 -- 172.21.15.82:0/1784404469 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4968 (secure 0 0 0) 0x7f704c015020 con 0x7f705c06eff0 2024-03-16T17:57:44.882 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:44.882 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2570794110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2570794110},{"type":"v1","addr":"172.21.15.90:6835","nonce":2570794110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3135739259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3135739259},{"type":"v1","addr":"172.21.15.82:6837","nonce":3135739259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3721205060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3721205060},{"type":"v1","addr":"172.21.15.82:6835","nonce":3721205060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:57:40.797669+0000","modified":"2024-03-16T17:57:43.865338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/732041173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":732041173},{"type":"v1","addr":"172.21.15.90:6833","nonce":732041173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4267,"qdb_cluster":[4267]},"id":1}]} 2024-03-16T17:57:44.882 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-03-16T17:57:44.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.880+0000 7f7062200640 1 -- 172.21.15.82:0/1784404469 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703806b3b0 msgr2=0x7f703806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:44.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.880+0000 7f7062200640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703806b3b0 0x7f703806d870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7044007fd0 tx=0x7f7044009040 comp rx=0 tx=0).stop 2024-03-16T17:57:44.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.880+0000 7f7062200640 1 -- 172.21.15.82:0/1784404469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06eff0 msgr2=0x7f705c160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:44.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.880+0000 7f7062200640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06eff0 0x7f705c160fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f704c00b3f0 tx=0x7f704c00b8b0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.881+0000 7f7062200640 1 -- 172.21.15.82:0/1784404469 shutdown_connections 2024-03-16T17:57:44.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.881+0000 7f7062200640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703806b3b0 0x7f703806d870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.881+0000 7f7062200640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f705c15e040 0x7f705c150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.881+0000 7f7062200640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f705c0d09a0 0x7f705c0b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.881+0000 7f7062200640 1 --2- 172.21.15.82:0/1784404469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06eff0 0x7f705c160fe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:44.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.881+0000 7f7062200640 1 -- 172.21.15.82:0/1784404469 >> 172.21.15.82:0/1784404469 conn(0x7f705c0b5060 msgr2=0x7f705c0af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:44.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.881+0000 7f7062200640 1 -- 172.21.15.82:0/1784404469 shutdown_connections 2024-03-16T17:57:44.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:44.881+0000 7f7062200640 1 -- 172.21.15.82:0/1784404469 wait complete. 2024-03-16T17:57:44.901 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:45.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.082+0000 7fd280a00640 1 Processor -- start 2024-03-16T17:57:45.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.082+0000 7fd280a00640 1 -- start start 2024-03-16T17:57:45.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.084+0000 7fd280a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd27c15e180 0x7fd27c0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.084+0000 7fd280a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd27c0d0e70 0x7fd27c0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.084+0000 7fd280a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd27c15e560 con 0x7fd27c15f110 2024-03-16T17:57:45.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.084+0000 7fd280a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd27c0cf0d0 con 0x7fd27c0d0e70 2024-03-16T17:57:45.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.084+0000 7fd280a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd27c0cf3b0 con 0x7fd27c15e180 2024-03-16T17:57:45.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.084+0000 7fd27a800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd27c0d0e70 0x7fd27c0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.084+0000 7fd27a800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd27c0d0e70 0x7fd27c0d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35062/0 (socket says 172.21.15.82:35062) 2024-03-16T17:57:45.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.084+0000 7fd27a800640 1 -- 172.21.15.82:0/2101428720 learned_addr learned my addr 172.21.15.82:0/2101428720 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:45.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.084+0000 7fd279e00640 1 --2- 172.21.15.82:0/2101428720 >> v2:172.21.15.90:3300/0 conn(0x7fd27c15e180 0x7fd27c0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.085+0000 7fd27a800640 1 -- 172.21.15.82:0/2101428720 >> v2:172.21.15.90:3300/0 conn(0x7fd27c15e180 msgr2=0x7fd27c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:45.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.085+0000 7fd27a800640 1 --2- 172.21.15.82:0/2101428720 >> v2:172.21.15.90:3300/0 conn(0x7fd27c15e180 0x7fd27c0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.085+0000 7fd27a800640 1 -- 172.21.15.82:0/2101428720 >> v1:172.21.15.82:6789/0 conn(0x7fd27c15f110 legacy=0x7fd27c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.085+0000 7fd279400640 1 -- 172.21.15.82:0/2101428720 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 210805998 0 0) 0x7fd264001110 con 0x7fd27c15f110 2024-03-16T17:57:45.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.085+0000 7fd27a800640 1 -- 172.21.15.82:0/2101428720 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd27c0cf690 con 0x7fd27c0d0e70 2024-03-16T17:57:45.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.085+0000 7fd279e00640 1 --2- 172.21.15.82:0/2101428720 >> v2:172.21.15.90:3300/0 conn(0x7fd27c15e180 0x7fd27c0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.086+0000 7fd27a800640 1 --2- 172.21.15.82:0/2101428720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 0x7fd27c0d1bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0b6a50 tx=0x7fd27402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee5810dbf628ded server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.086+0000 7fd279400640 1 -- 172.21.15.82:0/2101428720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27402cb20 con 0x7fd27c0d0e70 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.086+0000 7fd279400640 1 -- 172.21.15.82:0/2101428720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27402cc80 con 0x7fd27c0d0e70 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.086+0000 7fd279400640 1 -- 172.21.15.82:0/2101428720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2740343f0 con 0x7fd27c0d0e70 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.087+0000 7fd280a00640 1 -- 172.21.15.82:0/2101428720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 msgr2=0x7fd27c0d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.087+0000 7fd280a00640 1 --2- 172.21.15.82:0/2101428720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 0x7fd27c0d1bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd27c0b6a50 tx=0x7fd27402bf80 comp rx=0 tx=0).stop 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.087+0000 7fd280a00640 1 -- 172.21.15.82:0/2101428720 shutdown_connections 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.087+0000 7fd280a00640 1 --2- 172.21.15.82:0/2101428720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 0x7fd27c0d1bd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.087+0000 7fd280a00640 1 --2- 172.21.15.82:0/2101428720 >> v2:172.21.15.90:3300/0 conn(0x7fd27c15e180 0x7fd27c0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.087+0000 7fd280a00640 1 -- 172.21.15.82:0/2101428720 >> 172.21.15.82:0/2101428720 conn(0x7fd27c0d3e90 msgr2=0x7fd27c0d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.087+0000 7fd280a00640 1 -- 172.21.15.82:0/2101428720 shutdown_connections 2024-03-16T17:57:45.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.087+0000 7fd280a00640 1 -- 172.21.15.82:0/2101428720 wait complete. 2024-03-16T17:57:45.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.089+0000 7fd280a00640 1 Processor -- start 2024-03-16T17:57:45.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.089+0000 7fd280a00640 1 -- start start 2024-03-16T17:57:45.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd280a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 0x7fd27c1e4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd280a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd27c15e180 0x7fd27c14f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd280a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd27c15f110 0x7fd27c1e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd280a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c14f8b0 con 0x7fd27c0d0e70 2024-03-16T17:57:45.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd280a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c159500 con 0x7fd27c15f110 2024-03-16T17:57:45.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd280a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c1e4a80 con 0x7fd27c15e180 2024-03-16T17:57:45.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd279e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 0x7fd27c1e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd279e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 0x7fd27c1e4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35076/0 (socket says 172.21.15.82:35076) 2024-03-16T17:57:45.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd279e00640 1 -- 172.21.15.82:0/1051070933 learned_addr learned my addr 172.21.15.82:0/1051070933 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd27a800640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd27c15f110 0x7fd27c1e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd279e00640 1 -- 172.21.15.82:0/1051070933 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd27c15e180 msgr2=0x7fd27c14f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd271400640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd27c15e180 0x7fd27c14f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd279e00640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd27c15e180 0x7fd27c14f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd279e00640 1 -- 172.21.15.82:0/1051070933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd27c15f110 msgr2=0x7fd27c1e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd279e00640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd27c15f110 0x7fd27c1e5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd279e00640 1 -- 172.21.15.82:0/1051070933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274009b30 con 0x7fd27c0d0e70 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.090+0000 7fd27a800640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd27c15f110 0x7fd27c1e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.091+0000 7fd279e00640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 0x7fd27c1e4410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00e7e0 tx=0x7fd26c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.091+0000 7fd272a00640 1 -- 172.21.15.82:0/1051070933 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c004030 con 0x7fd27c0d0e70 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.091+0000 7fd272a00640 1 -- 172.21.15.82:0/1051070933 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26c004190 con 0x7fd27c0d0e70 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.091+0000 7fd272a00640 1 -- 172.21.15.82:0/1051070933 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c004520 con 0x7fd27c0d0e70 2024-03-16T17:57:45.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.091+0000 7fd280a00640 1 -- 172.21.15.82:0/1051070933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c1e4d60 con 0x7fd27c0d0e70 2024-03-16T17:57:45.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.091+0000 7fd280a00640 1 -- 172.21.15.82:0/1051070933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c1e5320 con 0x7fd27c0d0e70 2024-03-16T17:57:45.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.092+0000 7fd280a00640 1 -- 172.21.15.82:0/1051070933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27c067f30 con 0x7fd27c0d0e70 2024-03-16T17:57:45.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.093+0000 7fd272a00640 1 -- 172.21.15.82:0/1051070933 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fd26c00c6d0 con 0x7fd27c0d0e70 2024-03-16T17:57:45.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.094+0000 7fd272a00640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd25806b3b0 0x7fd25806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.094+0000 7fd272a00640 1 -- 172.21.15.82:0/1051070933 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd26c089840 con 0x7fd27c0d0e70 2024-03-16T17:57:45.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.095+0000 7fd271400640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd25806b3b0 0x7fd25806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.095+0000 7fd271400640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd25806b3b0 0x7fd25806d870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd27c1f6f80 tx=0x7fd264009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:45.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.099+0000 7fd272a00640 1 -- 172.21.15.82:0/1051070933 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd26c054530 con 0x7fd27c0d0e70 2024-03-16T17:57:45.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.280+0000 7fd280a00640 1 -- 172.21.15.82:0/1051070933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd27c08d8f0 con 0x7fd27c0d0e70 2024-03-16T17:57:45.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.283+0000 7fd272a00640 1 -- 172.21.15.82:0/1051070933 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fd26c057bf0 con 0x7fd27c0d0e70 2024-03-16T17:57:45.287 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:45.287 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:45.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.285+0000 7fd280a00640 1 -- 172.21.15.82:0/1051070933 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd25806b3b0 msgr2=0x7fd25806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.285+0000 7fd280a00640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd25806b3b0 0x7fd25806d870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd27c1f6f80 tx=0x7fd264009040 comp rx=0 tx=0).stop 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.285+0000 7fd280a00640 1 -- 172.21.15.82:0/1051070933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 msgr2=0x7fd27c1e4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.285+0000 7fd280a00640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 0x7fd27c1e4410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00e7e0 tx=0x7fd26c00eca0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.286+0000 7fd280a00640 1 -- 172.21.15.82:0/1051070933 shutdown_connections 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.286+0000 7fd280a00640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd25806b3b0 0x7fd25806d870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.286+0000 7fd280a00640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd27c15f110 0x7fd27c1e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.286+0000 7fd280a00640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd27c15e180 0x7fd27c14f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.286+0000 7fd280a00640 1 --2- 172.21.15.82:0/1051070933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd27c0d0e70 0x7fd27c1e4410 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.286+0000 7fd280a00640 1 -- 172.21.15.82:0/1051070933 >> 172.21.15.82:0/1051070933 conn(0x7fd27c0d3e90 msgr2=0x7fd27c0ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.286+0000 7fd280a00640 1 -- 172.21.15.82:0/1051070933 shutdown_connections 2024-03-16T17:57:45.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.286+0000 7fd280a00640 1 -- 172.21.15.82:0/1051070933 wait complete. 2024-03-16T17:57:45.307 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-16T17:57:45.307 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T17:57:45.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.508+0000 7f99d8000640 1 Processor -- start 2024-03-16T17:57:45.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.508+0000 7f99d8000640 1 -- start start 2024-03-16T17:57:45.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.510+0000 7f99d8000640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f99d01454d0 0x7f99d00b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.510+0000 7f99d8000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d00d0040 0x7f99d0070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.510+0000 7f99d8000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99d00b4b90 con 0x7f99d00cdf80 2024-03-16T17:57:45.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.510+0000 7f99d8000640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99d00cf500 con 0x7f99d01454d0 2024-03-16T17:57:45.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.510+0000 7f99d8000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d01458b0 con 0x7f99d00d0040 2024-03-16T17:57:45.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.510+0000 7f99d4c00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f99d00cdf80 0x7f99d00cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56140/0 (socket says 172.21.15.82:56140) 2024-03-16T17:57:45.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.510+0000 7f99d4c00640 1 -- 172.21.15.82:0/2099324120 learned_addr learned my addr 172.21.15.82:0/2099324120 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:45.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99d5600640 1 --2- 172.21.15.82:0/2099324120 >> v2:172.21.15.90:3300/0 conn(0x7f99d01454d0 0x7f99d00b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99d6000640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d00d0040 0x7f99d0070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126299259 0 0) 0x7f99d00b4b90 con 0x7f99d00cdf80 2024-03-16T17:57:45.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99b8003520 con 0x7f99d00cdf80 2024-03-16T17:57:45.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4042542476 0 0) 0x7f99b8003520 con 0x7f99d00cdf80 2024-03-16T17:57:45.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d00d0040 msgr2=0x7f99d0070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:45.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d00d0040 0x7f99d0070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 >> v2:172.21.15.90:3300/0 conn(0x7f99d01454d0 msgr2=0x7f99d00b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 --2- 172.21.15.82:0/2099324120 >> v2:172.21.15.90:3300/0 conn(0x7f99d01454d0 0x7f99d00b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d00ce460 con 0x7f99d00cdf80 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f99bc001670 con 0x7f99d00cdf80 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 >> v1:172.21.15.82:6789/0 conn(0x7f99d00cdf80 legacy=0x7f99d00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d00d0040 0x7f99b8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99b8003ef0 0x7f99b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b8004d00 0x7f99b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d00cf500 con 0x7f99b8004d00 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d01458b0 con 0x7f99b8003ef0 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.511+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b800b980 con 0x7f99d00d0040 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.512+0000 7f99d6000640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b8004d00 0x7f99b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.512+0000 7f99d4c00640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99b8003ef0 0x7f99b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.512+0000 7f99d5600640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d00d0040 0x7f99b8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.512+0000 7f99d6000640 1 -- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d00d0040 msgr2=0x7f99b8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.512+0000 7f99d6000640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d00d0040 0x7f99b8003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.512+0000 7f99d6000640 1 -- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99b8003ef0 msgr2=0x7f99b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.512+0000 7f99d6000640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99b8003ef0 0x7f99b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.512+0000 7f99d6000640 1 -- 172.21.15.82:0/2099324120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8003520 con 0x7f99b8004d00 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.513+0000 7f99d6000640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b8004d00 0x7f99b8005180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009f90 tx=0x7f99b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c798322c95ec4c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.513+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c802cbc0 con 0x7f99b8004d00 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.513+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c800f040 con 0x7f99b8004d00 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.513+0000 7f99cfe00640 1 -- 172.21.15.82:0/2099324120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c8034440 con 0x7f99b8004d00 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.514+0000 7f99d8000640 1 -- 172.21.15.82:0/2099324120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b8004d00 msgr2=0x7f99b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.514+0000 7f99d8000640 1 --2- 172.21.15.82:0/2099324120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99b8004d00 0x7f99b8005180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009f90 tx=0x7f99b80057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.514+0000 7f99d6000640 1 -- 172.21.15.82:0/2099324120 reap_dead start 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.514+0000 7f99d8000640 1 -- 172.21.15.82:0/2099324120 shutdown_connections 2024-03-16T17:57:45.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.514+0000 7f99d8000640 1 -- 172.21.15.82:0/2099324120 >> 172.21.15.82:0/2099324120 conn(0x7f99d006ae70 msgr2=0x7f99d006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:45.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.514+0000 7f99d8000640 1 -- 172.21.15.82:0/2099324120 shutdown_connections 2024-03-16T17:57:45.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.514+0000 7f99d8000640 1 -- 172.21.15.82:0/2099324120 wait complete. 2024-03-16T17:57:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.516+0000 7f99d8000640 1 Processor -- start 2024-03-16T17:57:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.516+0000 7f99d8000640 1 -- start start 2024-03-16T17:57:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d8000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d013cb10 0x7f99d0157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d8000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d013cef0 0x7f99d015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d8000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99d0158320 0x7f99d015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d8000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d01f5bf0 con 0x7f99d013cef0 2024-03-16T17:57:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d8000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99d01f4fe0 con 0x7f99d0158320 2024-03-16T17:57:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d8000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99d01f52c0 con 0x7f99d013cb10 2024-03-16T17:57:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d013cef0 0x7f99d015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d013cef0 0x7f99d015ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35098/0 (socket says 172.21.15.82:35098) 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d4c00640 1 -- 172.21.15.82:0/3378408292 learned_addr learned my addr 172.21.15.82:0/3378408292 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d5600640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d013cb10 0x7f99d0157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d6000640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99d0158320 0x7f99d015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d4c00640 1 -- 172.21.15.82:0/3378408292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d013cb10 msgr2=0x7f99d0157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d4c00640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d013cb10 0x7f99d0157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d4c00640 1 -- 172.21.15.82:0/3378408292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99d0158320 msgr2=0x7f99d015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d4c00640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99d0158320 0x7f99d015d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d4c00640 1 -- 172.21.15.82:0/3378408292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d01f55a0 con 0x7f99d013cef0 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.517+0000 7f99d4c00640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d013cef0 0x7f99d015ce30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00eb20 tx=0x7f99bc0016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.518+0000 7f99cd600640 1 -- 172.21.15.82:0/3378408292 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00c910 con 0x7f99d013cef0 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.518+0000 7f99cd600640 1 -- 172.21.15.82:0/3378408292 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc00ca70 con 0x7f99d013cef0 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.518+0000 7f99d8000640 1 -- 172.21.15.82:0/3378408292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99d01f5880 con 0x7f99d013cef0 2024-03-16T17:57:45.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.518+0000 7f99cd600640 1 -- 172.21.15.82:0/3378408292 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00ce00 con 0x7f99d013cef0 2024-03-16T17:57:45.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.518+0000 7f99d8000640 1 -- 172.21.15.82:0/3378408292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99d0158860 con 0x7f99d013cef0 2024-03-16T17:57:45.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.519+0000 7f99d8000640 1 -- 172.21.15.82:0/3378408292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99d0067f30 con 0x7f99d013cef0 2024-03-16T17:57:45.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.520+0000 7f99cd600640 1 -- 172.21.15.82:0/3378408292 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f99bc01d460 con 0x7f99d013cef0 2024-03-16T17:57:45.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.521+0000 7f99cd600640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99b006b3e0 0x7f99b006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.521+0000 7f99cd600640 1 -- 172.21.15.82:0/3378408292 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99bc089d10 con 0x7f99d013cef0 2024-03-16T17:57:45.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.521+0000 7f99d5600640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99b006b3e0 0x7f99b006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.522+0000 7f99d5600640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99b006b3e0 0x7f99b006d8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009990 tx=0x7f99c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:45.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.525+0000 7f99cd600640 1 -- 172.21.15.82:0/3378408292 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f99bc054a00 con 0x7f99d013cef0 2024-03-16T17:57:45.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.717+0000 7f99d8000640 1 -- 172.21.15.82:0/3378408292 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99d00ced00 con 0x7f99b006b3e0 2024-03-16T17:57:45.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.725+0000 7f99cd600640 1 -- 172.21.15.82:0/3378408292 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251947 (secure 0 0 0) 0x7f99c40118e0 con 0x7f99b006b3e0 2024-03-16T17:57:45.733 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:45.735 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T17:57:45.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 -- 172.21.15.82:0/3378408292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99b006b3e0 msgr2=0x7f99b006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99b006b3e0 0x7f99b006d8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009990 tx=0x7f99c4008040 comp rx=0 tx=0).stop 2024-03-16T17:57:45.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 -- 172.21.15.82:0/3378408292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d013cef0 msgr2=0x7f99d015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d013cef0 0x7f99d015ce30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00eb20 tx=0x7f99bc0016e0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 -- 172.21.15.82:0/3378408292 shutdown_connections 2024-03-16T17:57:45.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99b006b3e0 0x7f99b006d8a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99d0158320 0x7f99d015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99d013cef0 0x7f99d015ce30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 --2- 172.21.15.82:0/3378408292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99d013cb10 0x7f99d0157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 -- 172.21.15.82:0/3378408292 >> 172.21.15.82:0/3378408292 conn(0x7f99d006ae70 msgr2=0x7f99d00710d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:45.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.731+0000 7f99d8000640 1 -- 172.21.15.82:0/3378408292 shutdown_connections 2024-03-16T17:57:45.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.732+0000 7f99d8000640 1 -- 172.21.15.82:0/3378408292 wait complete. 2024-03-16T17:57:45.752 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-03-16T17:57:44.011085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219036,"kb_used_data":2840,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755684,"statfs":{"total":773094113280,"available":772869820416,"internally_reserved":0,"allocated":2908160,"data_stored":1753660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.680476"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736016+0000","last_change":"2024-03-16T17:57:41.412158+0000","last_active":"2024-03-16T17:57:42.736016+0000","last_peered":"2024-03-16T17:57:42.736016+0000","last_clean":"2024-03-16T17:57:42.736016+0000","last_became_active":"2024-03-16T17:57:41.411766+0000","last_became_peered":"2024-03-16T17:57:41.411766+0000","last_unstale":"2024-03-16T17:57:42.736016+0000","last_undegraded":"2024-03-16T17:57:42.736016+0000","last_fullsized":"2024-03-16T17:57:42.736016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:50:09.742348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370170+0000","last_change":"2024-03-16T17:57:41.383449+0000","last_active":"2024-03-16T17:57:42.370170+0000","last_peered":"2024-03-16T17:57:42.370170+0000","last_clean":"2024-03-16T17:57:42.370170+0000","last_became_active":"2024-03-16T17:57:41.382165+0000","last_became_peered":"2024-03-16T17:57:41.382165+0000","last_unstale":"2024-03-16T17:57:42.370170+0000","last_undegraded":"2024-03-16T17:57:42.370170+0000","last_fullsized":"2024-03-16T17:57:42.370170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732651+0000","last_change":"2024-03-16T17:57:41.380082+0000","last_active":"2024-03-16T17:57:42.732651+0000","last_peered":"2024-03-16T17:57:42.732651+0000","last_clean":"2024-03-16T17:57:42.732651+0000","last_became_active":"2024-03-16T17:57:41.379508+0000","last_became_peered":"2024-03-16T17:57:41.379508+0000","last_unstale":"2024-03-16T17:57:42.732651+0000","last_undegraded":"2024-03-16T17:57:42.732651+0000","last_fullsized":"2024-03-16T17:57:42.732651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375083+0000","last_change":"2024-03-16T17:57:41.404364+0000","last_active":"2024-03-16T17:57:42.375083+0000","last_peered":"2024-03-16T17:57:42.375083+0000","last_clean":"2024-03-16T17:57:42.375083+0000","last_became_active":"2024-03-16T17:57:41.404048+0000","last_became_peered":"2024-03-16T17:57:41.404048+0000","last_unstale":"2024-03-16T17:57:42.375083+0000","last_undegraded":"2024-03-16T17:57:42.375083+0000","last_fullsized":"2024-03-16T17:57:42.375083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734821+0000","last_change":"2024-03-16T17:57:41.392327+0000","last_active":"2024-03-16T17:57:42.734821+0000","last_peered":"2024-03-16T17:57:42.734821+0000","last_clean":"2024-03-16T17:57:42.734821+0000","last_became_active":"2024-03-16T17:57:41.392007+0000","last_became_peered":"2024-03-16T17:57:41.392007+0000","last_unstale":"2024-03-16T17:57:42.734821+0000","last_undegraded":"2024-03-16T17:57:42.734821+0000","last_fullsized":"2024-03-16T17:57:42.734821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:48:10.857666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375324+0000","last_change":"2024-03-16T17:57:41.398446+0000","last_active":"2024-03-16T17:57:42.375324+0000","last_peered":"2024-03-16T17:57:42.375324+0000","last_clean":"2024-03-16T17:57:42.375324+0000","last_became_active":"2024-03-16T17:57:41.398135+0000","last_became_peered":"2024-03-16T17:57:41.398135+0000","last_unstale":"2024-03-16T17:57:42.375324+0000","last_undegraded":"2024-03-16T17:57:42.375324+0000","last_fullsized":"2024-03-16T17:57:42.375324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:48:10.857666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734440+0000","last_change":"2024-03-16T17:57:41.405535+0000","last_active":"2024-03-16T17:57:42.734440+0000","last_peered":"2024-03-16T17:57:42.734440+0000","last_clean":"2024-03-16T17:57:42.734440+0000","last_became_active":"2024-03-16T17:57:41.405323+0000","last_became_peered":"2024-03-16T17:57:41.405323+0000","last_unstale":"2024-03-16T17:57:42.734440+0000","last_undegraded":"2024-03-16T17:57:42.734440+0000","last_fullsized":"2024-03-16T17:57:42.734440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.731940+0000","last_change":"2024-03-16T17:57:41.402459+0000","last_active":"2024-03-16T17:57:42.731940+0000","last_peered":"2024-03-16T17:57:42.731940+0000","last_clean":"2024-03-16T17:57:42.731940+0000","last_became_active":"2024-03-16T17:57:41.402103+0000","last_became_peered":"2024-03-16T17:57:41.402103+0000","last_unstale":"2024-03-16T17:57:42.731940+0000","last_undegraded":"2024-03-16T17:57:42.731940+0000","last_fullsized":"2024-03-16T17:57:42.731940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:20.628509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736292+0000","last_change":"2024-03-16T17:57:41.409011+0000","last_active":"2024-03-16T17:57:42.736292+0000","last_peered":"2024-03-16T17:57:42.736292+0000","last_clean":"2024-03-16T17:57:42.736292+0000","last_became_active":"2024-03-16T17:57:41.408811+0000","last_became_peered":"2024-03-16T17:57:41.408811+0000","last_unstale":"2024-03-16T17:57:42.736292+0000","last_undegraded":"2024-03-16T17:57:42.736292+0000","last_fullsized":"2024-03-16T17:57:42.736292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:11:48.511692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734591+0000","last_change":"2024-03-16T17:57:41.401707+0000","last_active":"2024-03-16T17:57:42.734591+0000","last_peered":"2024-03-16T17:57:42.734591+0000","last_clean":"2024-03-16T17:57:42.734591+0000","last_became_active":"2024-03-16T17:57:41.401330+0000","last_became_peered":"2024-03-16T17:57:41.401330+0000","last_unstale":"2024-03-16T17:57:42.734591+0000","last_undegraded":"2024-03-16T17:57:42.734591+0000","last_fullsized":"2024-03-16T17:57:42.734591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733619+0000","last_change":"2024-03-16T17:57:41.399956+0000","last_active":"2024-03-16T17:57:42.733619+0000","last_peered":"2024-03-16T17:57:42.733619+0000","last_clean":"2024-03-16T17:57:42.733619+0000","last_became_active":"2024-03-16T17:57:41.399696+0000","last_became_peered":"2024-03-16T17:57:41.399696+0000","last_unstale":"2024-03-16T17:57:42.733619+0000","last_undegraded":"2024-03-16T17:57:42.733619+0000","last_fullsized":"2024-03-16T17:57:42.733619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373739+0000","last_change":"2024-03-16T17:57:41.397224+0000","last_active":"2024-03-16T17:57:42.373739+0000","last_peered":"2024-03-16T17:57:42.373739+0000","last_clean":"2024-03-16T17:57:42.373739+0000","last_became_active":"2024-03-16T17:57:41.396857+0000","last_became_peered":"2024-03-16T17:57:41.396857+0000","last_unstale":"2024-03-16T17:57:42.373739+0000","last_undegraded":"2024-03-16T17:57:42.373739+0000","last_fullsized":"2024-03-16T17:57:42.373739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:32:52.837630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734251+0000","last_change":"2024-03-16T17:57:41.399774+0000","last_active":"2024-03-16T17:57:42.734251+0000","last_peered":"2024-03-16T17:57:42.734251+0000","last_clean":"2024-03-16T17:57:42.734251+0000","last_became_active":"2024-03-16T17:57:41.399505+0000","last_became_peered":"2024-03-16T17:57:41.399505+0000","last_unstale":"2024-03-16T17:57:42.734251+0000","last_undegraded":"2024-03-16T17:57:42.734251+0000","last_fullsized":"2024-03-16T17:57:42.734251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732264+0000","last_change":"2024-03-16T17:57:41.380132+0000","last_active":"2024-03-16T17:57:42.732264+0000","last_peered":"2024-03-16T17:57:42.732264+0000","last_clean":"2024-03-16T17:57:42.732264+0000","last_became_active":"2024-03-16T17:57:41.379536+0000","last_became_peered":"2024-03-16T17:57:41.379536+0000","last_unstale":"2024-03-16T17:57:42.732264+0000","last_undegraded":"2024-03-16T17:57:42.732264+0000","last_fullsized":"2024-03-16T17:57:42.732264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:54:00.468586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369395+0000","last_change":"2024-03-16T17:57:41.393322+0000","last_active":"2024-03-16T17:57:42.369395+0000","last_peered":"2024-03-16T17:57:42.369395+0000","last_clean":"2024-03-16T17:57:42.369395+0000","last_became_active":"2024-03-16T17:57:41.391272+0000","last_became_peered":"2024-03-16T17:57:41.391272+0000","last_unstale":"2024-03-16T17:57:42.369395+0000","last_undegraded":"2024-03-16T17:57:42.369395+0000","last_fullsized":"2024-03-16T17:57:42.369395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374104+0000","last_change":"2024-03-16T17:57:41.399992+0000","last_active":"2024-03-16T17:57:42.374104+0000","last_peered":"2024-03-16T17:57:42.374104+0000","last_clean":"2024-03-16T17:57:42.374104+0000","last_became_active":"2024-03-16T17:57:41.399406+0000","last_became_peered":"2024-03-16T17:57:41.399406+0000","last_unstale":"2024-03-16T17:57:42.374104+0000","last_undegraded":"2024-03-16T17:57:42.374104+0000","last_fullsized":"2024-03-16T17:57:42.374104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376607+0000","last_change":"2024-03-16T17:57:41.416986+0000","last_active":"2024-03-16T17:57:42.376607+0000","last_peered":"2024-03-16T17:57:42.376607+0000","last_clean":"2024-03-16T17:57:42.376607+0000","last_became_active":"2024-03-16T17:57:41.416699+0000","last_became_peered":"2024-03-16T17:57:41.416699+0000","last_unstale":"2024-03-16T17:57:42.376607+0000","last_undegraded":"2024-03-16T17:57:42.376607+0000","last_fullsized":"2024-03-16T17:57:42.376607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:11:48.511692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733913+0000","last_change":"2024-03-16T17:57:41.394311+0000","last_active":"2024-03-16T17:57:42.733913+0000","last_peered":"2024-03-16T17:57:42.733913+0000","last_clean":"2024-03-16T17:57:42.733913+0000","last_became_active":"2024-03-16T17:57:41.393750+0000","last_became_peered":"2024-03-16T17:57:41.393750+0000","last_unstale":"2024-03-16T17:57:42.733913+0000","last_undegraded":"2024-03-16T17:57:42.733913+0000","last_fullsized":"2024-03-16T17:57:42.733913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:55:54.818895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736015+0000","last_change":"2024-03-16T17:57:41.408696+0000","last_active":"2024-03-16T17:57:42.736015+0000","last_peered":"2024-03-16T17:57:42.736015+0000","last_clean":"2024-03-16T17:57:42.736015+0000","last_became_active":"2024-03-16T17:57:41.407916+0000","last_became_peered":"2024-03-16T17:57:41.407916+0000","last_unstale":"2024-03-16T17:57:42.736015+0000","last_undegraded":"2024-03-16T17:57:42.736015+0000","last_fullsized":"2024-03-16T17:57:42.736015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:36:37.396828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733788+0000","last_change":"2024-03-16T17:57:41.396339+0000","last_active":"2024-03-16T17:57:42.733788+0000","last_peered":"2024-03-16T17:57:42.733788+0000","last_clean":"2024-03-16T17:57:42.733788+0000","last_became_active":"2024-03-16T17:57:41.396068+0000","last_became_peered":"2024-03-16T17:57:41.396068+0000","last_unstale":"2024-03-16T17:57:42.733788+0000","last_undegraded":"2024-03-16T17:57:42.733788+0000","last_fullsized":"2024-03-16T17:57:42.733788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732607+0000","last_change":"2024-03-16T17:57:41.390555+0000","last_active":"2024-03-16T17:57:42.732607+0000","last_peered":"2024-03-16T17:57:42.732607+0000","last_clean":"2024-03-16T17:57:42.732607+0000","last_became_active":"2024-03-16T17:57:41.389886+0000","last_became_peered":"2024-03-16T17:57:41.389886+0000","last_unstale":"2024-03-16T17:57:42.732607+0000","last_undegraded":"2024-03-16T17:57:42.732607+0000","last_fullsized":"2024-03-16T17:57:42.732607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:20.628509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375544+0000","last_change":"2024-03-16T17:57:41.411237+0000","last_active":"2024-03-16T17:57:42.375544+0000","last_peered":"2024-03-16T17:57:42.375544+0000","last_clean":"2024-03-16T17:57:42.375544+0000","last_became_active":"2024-03-16T17:57:41.410757+0000","last_became_peered":"2024-03-16T17:57:41.410757+0000","last_unstale":"2024-03-16T17:57:42.375544+0000","last_undegraded":"2024-03-16T17:57:42.375544+0000","last_fullsized":"2024-03-16T17:57:42.375544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:01:51.467418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733707+0000","last_change":"2024-03-16T17:57:41.391770+0000","last_active":"2024-03-16T17:57:42.733707+0000","last_peered":"2024-03-16T17:57:42.733707+0000","last_clean":"2024-03-16T17:57:42.733707+0000","last_became_active":"2024-03-16T17:57:41.391470+0000","last_became_peered":"2024-03-16T17:57:41.391470+0000","last_unstale":"2024-03-16T17:57:42.733707+0000","last_undegraded":"2024-03-16T17:57:42.733707+0000","last_fullsized":"2024-03-16T17:57:42.733707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374329+0000","last_change":"2024-03-16T17:57:41.400595+0000","last_active":"2024-03-16T17:57:42.374329+0000","last_peered":"2024-03-16T17:57:42.374329+0000","last_clean":"2024-03-16T17:57:42.374329+0000","last_became_active":"2024-03-16T17:57:41.400295+0000","last_became_peered":"2024-03-16T17:57:41.400295+0000","last_unstale":"2024-03-16T17:57:42.374329+0000","last_undegraded":"2024-03-16T17:57:42.374329+0000","last_fullsized":"2024-03-16T17:57:42.374329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:55:42.207131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734977+0000","last_change":"2024-03-16T17:57:41.391182+0000","last_active":"2024-03-16T17:57:42.734977+0000","last_peered":"2024-03-16T17:57:42.734977+0000","last_clean":"2024-03-16T17:57:42.734977+0000","last_became_active":"2024-03-16T17:57:41.390811+0000","last_became_peered":"2024-03-16T17:57:41.390811+0000","last_unstale":"2024-03-16T17:57:42.734977+0000","last_undegraded":"2024-03-16T17:57:42.734977+0000","last_fullsized":"2024-03-16T17:57:42.734977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:55:42.207131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734433+0000","last_change":"2024-03-16T17:57:41.407541+0000","last_active":"2024-03-16T17:57:42.734433+0000","last_peered":"2024-03-16T17:57:42.734433+0000","last_clean":"2024-03-16T17:57:42.734433+0000","last_became_active":"2024-03-16T17:57:41.405546+0000","last_became_peered":"2024-03-16T17:57:41.405546+0000","last_unstale":"2024-03-16T17:57:42.734433+0000","last_undegraded":"2024-03-16T17:57:42.734433+0000","last_fullsized":"2024-03-16T17:57:42.734433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369831+0000","last_change":"2024-03-16T17:57:41.399134+0000","last_active":"2024-03-16T17:57:42.369831+0000","last_peered":"2024-03-16T17:57:42.369831+0000","last_clean":"2024-03-16T17:57:42.369831+0000","last_became_active":"2024-03-16T17:57:41.398841+0000","last_became_peered":"2024-03-16T17:57:41.398841+0000","last_unstale":"2024-03-16T17:57:42.369831+0000","last_undegraded":"2024-03-16T17:57:42.369831+0000","last_fullsized":"2024-03-16T17:57:42.369831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734641+0000","last_change":"2024-03-16T17:57:41.399114+0000","last_active":"2024-03-16T17:57:42.734641+0000","last_peered":"2024-03-16T17:57:42.734641+0000","last_clean":"2024-03-16T17:57:42.734641+0000","last_became_active":"2024-03-16T17:57:41.395668+0000","last_became_peered":"2024-03-16T17:57:41.395668+0000","last_unstale":"2024-03-16T17:57:42.734641+0000","last_undegraded":"2024-03-16T17:57:42.734641+0000","last_fullsized":"2024-03-16T17:57:42.734641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732560+0000","last_change":"2024-03-16T17:57:41.390678+0000","last_active":"2024-03-16T17:57:42.732560+0000","last_peered":"2024-03-16T17:57:42.732560+0000","last_clean":"2024-03-16T17:57:42.732560+0000","last_became_active":"2024-03-16T17:57:41.387968+0000","last_became_peered":"2024-03-16T17:57:41.387968+0000","last_unstale":"2024-03-16T17:57:42.732560+0000","last_undegraded":"2024-03-16T17:57:42.732560+0000","last_fullsized":"2024-03-16T17:57:42.732560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:59:12.277190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375711+0000","last_change":"2024-03-16T17:57:41.399083+0000","last_active":"2024-03-16T17:57:42.375711+0000","last_peered":"2024-03-16T17:57:42.375711+0000","last_clean":"2024-03-16T17:57:42.375711+0000","last_became_active":"2024-03-16T17:57:41.398899+0000","last_became_peered":"2024-03-16T17:57:41.398899+0000","last_unstale":"2024-03-16T17:57:42.375711+0000","last_undegraded":"2024-03-16T17:57:42.375711+0000","last_fullsized":"2024-03-16T17:57:42.375711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:10:39.722613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733367+0000","last_change":"2024-03-16T17:57:41.390700+0000","last_active":"2024-03-16T17:57:42.733367+0000","last_peered":"2024-03-16T17:57:42.733367+0000","last_clean":"2024-03-16T17:57:42.733367+0000","last_became_active":"2024-03-16T17:57:41.389732+0000","last_became_peered":"2024-03-16T17:57:41.389732+0000","last_unstale":"2024-03-16T17:57:42.733367+0000","last_undegraded":"2024-03-16T17:57:42.733367+0000","last_fullsized":"2024-03-16T17:57:42.733367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:32:52.837630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732990+0000","last_change":"2024-03-16T17:57:41.392698+0000","last_active":"2024-03-16T17:57:42.732990+0000","last_peered":"2024-03-16T17:57:42.732990+0000","last_clean":"2024-03-16T17:57:42.732990+0000","last_became_active":"2024-03-16T17:57:41.391950+0000","last_became_peered":"2024-03-16T17:57:41.391950+0000","last_unstale":"2024-03-16T17:57:42.732990+0000","last_undegraded":"2024-03-16T17:57:42.732990+0000","last_fullsized":"2024-03-16T17:57:42.732990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.372924+0000","last_change":"2024-03-16T17:57:41.383046+0000","last_active":"2024-03-16T17:57:42.372924+0000","last_peered":"2024-03-16T17:57:42.372924+0000","last_clean":"2024-03-16T17:57:42.372924+0000","last_became_active":"2024-03-16T17:57:41.382533+0000","last_became_peered":"2024-03-16T17:57:41.382533+0000","last_unstale":"2024-03-16T17:57:42.372924+0000","last_undegraded":"2024-03-16T17:57:42.372924+0000","last_fullsized":"2024-03-16T17:57:42.372924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:44:11.757790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734496+0000","last_change":"2024-03-16T17:57:40.368183+0000","last_active":"2024-03-16T17:57:42.734496+0000","last_peered":"2024-03-16T17:57:42.734496+0000","last_clean":"2024-03-16T17:57:42.734496+0000","last_became_active":"2024-03-16T17:57:40.367738+0000","last_became_peered":"2024-03-16T17:57:40.367738+0000","last_unstale":"2024-03-16T17:57:42.734496+0000","last_undegraded":"2024-03-16T17:57:42.734496+0000","last_fullsized":"2024-03-16T17:57:42.734496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732742+0000","last_change":"2024-03-16T17:57:41.378107+0000","last_active":"2024-03-16T17:57:42.732742+0000","last_peered":"2024-03-16T17:57:42.732742+0000","last_clean":"2024-03-16T17:57:42.732742+0000","last_became_active":"2024-03-16T17:57:41.375872+0000","last_became_peered":"2024-03-16T17:57:41.375872+0000","last_unstale":"2024-03-16T17:57:42.732742+0000","last_undegraded":"2024-03-16T17:57:42.732742+0000","last_fullsized":"2024-03-16T17:57:42.732742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733377+0000","last_change":"2024-03-16T17:57:40.375279+0000","last_active":"2024-03-16T17:57:42.733377+0000","last_peered":"2024-03-16T17:57:42.733377+0000","last_clean":"2024-03-16T17:57:42.733377+0000","last_became_active":"2024-03-16T17:57:40.374936+0000","last_became_peered":"2024-03-16T17:57:40.374936+0000","last_unstale":"2024-03-16T17:57:42.733377+0000","last_undegraded":"2024-03-16T17:57:42.733377+0000","last_fullsized":"2024-03-16T17:57:42.733377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:07.628213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733869+0000","last_change":"2024-03-16T17:57:40.372178+0000","last_active":"2024-03-16T17:57:42.733869+0000","last_peered":"2024-03-16T17:57:42.733869+0000","last_clean":"2024-03-16T17:57:42.733869+0000","last_became_active":"2024-03-16T17:57:40.370709+0000","last_became_peered":"2024-03-16T17:57:40.370709+0000","last_unstale":"2024-03-16T17:57:42.733869+0000","last_undegraded":"2024-03-16T17:57:42.733869+0000","last_fullsized":"2024-03-16T17:57:42.733869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:23:36.985797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732131+0000","last_change":"2024-03-16T17:57:41.397372+0000","last_active":"2024-03-16T17:57:42.732131+0000","last_peered":"2024-03-16T17:57:42.732131+0000","last_clean":"2024-03-16T17:57:42.732131+0000","last_became_active":"2024-03-16T17:57:41.397032+0000","last_became_peered":"2024-03-16T17:57:41.397032+0000","last_unstale":"2024-03-16T17:57:42.732131+0000","last_undegraded":"2024-03-16T17:57:42.732131+0000","last_fullsized":"2024-03-16T17:57:42.732131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374381+0000","last_change":"2024-03-16T17:57:40.362691+0000","last_active":"2024-03-16T17:57:42.374381+0000","last_peered":"2024-03-16T17:57:42.374381+0000","last_clean":"2024-03-16T17:57:42.374381+0000","last_became_active":"2024-03-16T17:57:40.361668+0000","last_became_peered":"2024-03-16T17:57:40.361668+0000","last_unstale":"2024-03-16T17:57:42.374381+0000","last_undegraded":"2024-03-16T17:57:42.374381+0000","last_fullsized":"2024-03-16T17:57:42.374381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369126+0000","last_change":"2024-03-16T17:57:41.393663+0000","last_active":"2024-03-16T17:57:42.369126+0000","last_peered":"2024-03-16T17:57:42.369126+0000","last_clean":"2024-03-16T17:57:42.369126+0000","last_became_active":"2024-03-16T17:57:41.393327+0000","last_became_peered":"2024-03-16T17:57:41.393327+0000","last_unstale":"2024-03-16T17:57:42.369126+0000","last_undegraded":"2024-03-16T17:57:42.369126+0000","last_fullsized":"2024-03-16T17:57:42.369126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733784+0000","last_change":"2024-03-16T17:57:40.377710+0000","last_active":"2024-03-16T17:57:42.733784+0000","last_peered":"2024-03-16T17:57:42.733784+0000","last_clean":"2024-03-16T17:57:42.733784+0000","last_became_active":"2024-03-16T17:57:40.377427+0000","last_became_peered":"2024-03-16T17:57:40.377427+0000","last_unstale":"2024-03-16T17:57:42.733784+0000","last_undegraded":"2024-03-16T17:57:42.733784+0000","last_fullsized":"2024-03-16T17:57:42.733784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376320+0000","last_change":"2024-03-16T17:57:41.413272+0000","last_active":"2024-03-16T17:57:42.376320+0000","last_peered":"2024-03-16T17:57:42.376320+0000","last_clean":"2024-03-16T17:57:42.376320+0000","last_became_active":"2024-03-16T17:57:41.412989+0000","last_became_peered":"2024-03-16T17:57:41.412989+0000","last_unstale":"2024-03-16T17:57:42.376320+0000","last_undegraded":"2024-03-16T17:57:42.376320+0000","last_fullsized":"2024-03-16T17:57:42.376320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:36:37.396828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376311+0000","last_change":"2024-03-16T17:57:40.380326+0000","last_active":"2024-03-16T17:57:42.376311+0000","last_peered":"2024-03-16T17:57:42.376311+0000","last_clean":"2024-03-16T17:57:42.376311+0000","last_became_active":"2024-03-16T17:57:40.380003+0000","last_became_peered":"2024-03-16T17:57:40.380003+0000","last_unstale":"2024-03-16T17:57:42.376311+0000","last_undegraded":"2024-03-16T17:57:42.376311+0000","last_fullsized":"2024-03-16T17:57:42.376311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:07.628213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733226+0000","last_change":"2024-03-16T17:57:41.407675+0000","last_active":"2024-03-16T17:57:42.733226+0000","last_peered":"2024-03-16T17:57:42.733226+0000","last_clean":"2024-03-16T17:57:42.733226+0000","last_became_active":"2024-03-16T17:57:41.407412+0000","last_became_peered":"2024-03-16T17:57:41.407412+0000","last_unstale":"2024-03-16T17:57:42.733226+0000","last_undegraded":"2024-03-16T17:57:42.733226+0000","last_fullsized":"2024-03-16T17:57:42.733226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370065+0000","last_change":"2024-03-16T17:57:40.375361+0000","last_active":"2024-03-16T17:57:42.370065+0000","last_peered":"2024-03-16T17:57:42.370065+0000","last_clean":"2024-03-16T17:57:42.370065+0000","last_became_active":"2024-03-16T17:57:40.375086+0000","last_became_peered":"2024-03-16T17:57:40.375086+0000","last_unstale":"2024-03-16T17:57:42.370065+0000","last_undegraded":"2024-03-16T17:57:42.370065+0000","last_fullsized":"2024-03-16T17:57:42.370065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373846+0000","last_change":"2024-03-16T17:57:41.404551+0000","last_active":"2024-03-16T17:57:42.373846+0000","last_peered":"2024-03-16T17:57:42.373846+0000","last_clean":"2024-03-16T17:57:42.373846+0000","last_became_active":"2024-03-16T17:57:41.404266+0000","last_became_peered":"2024-03-16T17:57:41.404266+0000","last_unstale":"2024-03-16T17:57:42.373846+0000","last_undegraded":"2024-03-16T17:57:42.373846+0000","last_fullsized":"2024-03-16T17:57:42.373846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:55:54.818895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733842+0000","last_change":"2024-03-16T17:57:40.376223+0000","last_active":"2024-03-16T17:57:42.733842+0000","last_peered":"2024-03-16T17:57:42.733842+0000","last_clean":"2024-03-16T17:57:42.733842+0000","last_became_active":"2024-03-16T17:57:40.375873+0000","last_became_peered":"2024-03-16T17:57:40.375873+0000","last_unstale":"2024-03-16T17:57:42.733842+0000","last_undegraded":"2024-03-16T17:57:42.733842+0000","last_fullsized":"2024-03-16T17:57:42.733842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735456+0000","last_change":"2024-03-16T17:57:41.403063+0000","last_active":"2024-03-16T17:57:42.735456+0000","last_peered":"2024-03-16T17:57:42.735456+0000","last_clean":"2024-03-16T17:57:42.735456+0000","last_became_active":"2024-03-16T17:57:41.402736+0000","last_became_peered":"2024-03-16T17:57:41.402736+0000","last_unstale":"2024-03-16T17:57:42.735456+0000","last_undegraded":"2024-03-16T17:57:42.735456+0000","last_fullsized":"2024-03-16T17:57:42.735456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:10:39.722613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732825+0000","last_change":"2024-03-16T17:57:40.363660+0000","last_active":"2024-03-16T17:57:42.732825+0000","last_peered":"2024-03-16T17:57:42.732825+0000","last_clean":"2024-03-16T17:57:42.732825+0000","last_became_active":"2024-03-16T17:57:40.363245+0000","last_became_peered":"2024-03-16T17:57:40.363245+0000","last_unstale":"2024-03-16T17:57:42.732825+0000","last_undegraded":"2024-03-16T17:57:42.732825+0000","last_fullsized":"2024-03-16T17:57:42.732825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:45:57.586553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732613+0000","last_change":"2024-03-16T17:57:41.398106+0000","last_active":"2024-03-16T17:57:42.732613+0000","last_peered":"2024-03-16T17:57:42.732613+0000","last_clean":"2024-03-16T17:57:42.732613+0000","last_became_active":"2024-03-16T17:57:41.396446+0000","last_became_peered":"2024-03-16T17:57:41.396446+0000","last_unstale":"2024-03-16T17:57:42.732613+0000","last_undegraded":"2024-03-16T17:57:42.732613+0000","last_fullsized":"2024-03-16T17:57:42.732613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:42:47.316038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375786+0000","last_change":"2024-03-16T17:57:40.377535+0000","last_active":"2024-03-16T17:57:42.375786+0000","last_peered":"2024-03-16T17:57:42.375786+0000","last_clean":"2024-03-16T17:57:42.375786+0000","last_became_active":"2024-03-16T17:57:40.377105+0000","last_became_peered":"2024-03-16T17:57:40.377105+0000","last_unstale":"2024-03-16T17:57:42.375786+0000","last_undegraded":"2024-03-16T17:57:42.375786+0000","last_fullsized":"2024-03-16T17:57:42.375786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735111+0000","last_change":"2024-03-16T17:57:41.407322+0000","last_active":"2024-03-16T17:57:42.735111+0000","last_peered":"2024-03-16T17:57:42.735111+0000","last_clean":"2024-03-16T17:57:42.735111+0000","last_became_active":"2024-03-16T17:57:41.403879+0000","last_became_peered":"2024-03-16T17:57:41.403879+0000","last_unstale":"2024-03-16T17:57:42.735111+0000","last_undegraded":"2024-03-16T17:57:42.735111+0000","last_fullsized":"2024-03-16T17:57:42.735111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:16:17.454713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732837+0000","last_change":"2024-03-16T17:57:40.368238+0000","last_active":"2024-03-16T17:57:42.732837+0000","last_peered":"2024-03-16T17:57:42.732837+0000","last_clean":"2024-03-16T17:57:42.732837+0000","last_became_active":"2024-03-16T17:57:40.367367+0000","last_became_peered":"2024-03-16T17:57:40.367367+0000","last_unstale":"2024-03-16T17:57:42.732837+0000","last_undegraded":"2024-03-16T17:57:42.732837+0000","last_fullsized":"2024-03-16T17:57:42.732837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:26.104046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732994+0000","last_change":"2024-03-16T17:57:41.389244+0000","last_active":"2024-03-16T17:57:42.732994+0000","last_peered":"2024-03-16T17:57:42.732994+0000","last_clean":"2024-03-16T17:57:42.732994+0000","last_became_active":"2024-03-16T17:57:41.388858+0000","last_became_peered":"2024-03-16T17:57:41.388858+0000","last_unstale":"2024-03-16T17:57:42.732994+0000","last_undegraded":"2024-03-16T17:57:42.732994+0000","last_fullsized":"2024-03-16T17:57:42.732994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733311+0000","last_change":"2024-03-16T17:57:40.365096+0000","last_active":"2024-03-16T17:57:42.733311+0000","last_peered":"2024-03-16T17:57:42.733311+0000","last_clean":"2024-03-16T17:57:42.733311+0000","last_became_active":"2024-03-16T17:57:40.364700+0000","last_became_peered":"2024-03-16T17:57:40.364700+0000","last_unstale":"2024-03-16T17:57:42.733311+0000","last_undegraded":"2024-03-16T17:57:42.733311+0000","last_fullsized":"2024-03-16T17:57:42.733311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:53:51.562074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734633+0000","last_change":"2024-03-16T17:57:41.405056+0000","last_active":"2024-03-16T17:57:42.734633+0000","last_peered":"2024-03-16T17:57:42.734633+0000","last_clean":"2024-03-16T17:57:42.734633+0000","last_became_active":"2024-03-16T17:57:41.403286+0000","last_became_peered":"2024-03-16T17:57:41.403286+0000","last_unstale":"2024-03-16T17:57:42.734633+0000","last_undegraded":"2024-03-16T17:57:42.734633+0000","last_fullsized":"2024-03-16T17:57:42.734633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374263+0000","last_change":"2024-03-16T17:57:40.365256+0000","last_active":"2024-03-16T17:57:42.374263+0000","last_peered":"2024-03-16T17:57:42.374263+0000","last_clean":"2024-03-16T17:57:42.374263+0000","last_became_active":"2024-03-16T17:57:40.362214+0000","last_became_peered":"2024-03-16T17:57:40.362214+0000","last_unstale":"2024-03-16T17:57:42.374263+0000","last_undegraded":"2024-03-16T17:57:42.374263+0000","last_fullsized":"2024-03-16T17:57:42.374263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:22.039976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735167+0000","last_change":"2024-03-16T17:57:41.413401+0000","last_active":"2024-03-16T17:57:42.735167+0000","last_peered":"2024-03-16T17:57:42.735167+0000","last_clean":"2024-03-16T17:57:42.735167+0000","last_became_active":"2024-03-16T17:57:41.413163+0000","last_became_peered":"2024-03-16T17:57:41.413163+0000","last_unstale":"2024-03-16T17:57:42.735167+0000","last_undegraded":"2024-03-16T17:57:42.735167+0000","last_fullsized":"2024-03-16T17:57:42.735167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:01:51.467418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734097+0000","last_change":"2024-03-16T17:57:40.370143+0000","last_active":"2024-03-16T17:57:42.734097+0000","last_peered":"2024-03-16T17:57:42.734097+0000","last_clean":"2024-03-16T17:57:42.734097+0000","last_became_active":"2024-03-16T17:57:40.369743+0000","last_became_peered":"2024-03-16T17:57:40.369743+0000","last_unstale":"2024-03-16T17:57:42.734097+0000","last_undegraded":"2024-03-16T17:57:42.734097+0000","last_fullsized":"2024-03-16T17:57:42.734097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:32:24.057779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736551+0000","last_change":"2024-03-16T17:57:41.413116+0000","last_active":"2024-03-16T17:57:42.736551+0000","last_peered":"2024-03-16T17:57:42.736551+0000","last_clean":"2024-03-16T17:57:42.736551+0000","last_became_active":"2024-03-16T17:57:41.412868+0000","last_became_peered":"2024-03-16T17:57:41.412868+0000","last_unstale":"2024-03-16T17:57:42.736551+0000","last_undegraded":"2024-03-16T17:57:42.736551+0000","last_fullsized":"2024-03-16T17:57:42.736551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:20:56.534791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370370+0000","last_change":"2024-03-16T17:57:40.364807+0000","last_active":"2024-03-16T17:57:42.370370+0000","last_peered":"2024-03-16T17:57:42.370370+0000","last_clean":"2024-03-16T17:57:42.370370+0000","last_became_active":"2024-03-16T17:57:40.363383+0000","last_became_peered":"2024-03-16T17:57:40.363383+0000","last_unstale":"2024-03-16T17:57:42.370370+0000","last_undegraded":"2024-03-16T17:57:42.370370+0000","last_fullsized":"2024-03-16T17:57:42.370370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:16.023603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375496+0000","last_change":"2024-03-16T17:57:41.402990+0000","last_active":"2024-03-16T17:57:42.375496+0000","last_peered":"2024-03-16T17:57:42.375496+0000","last_clean":"2024-03-16T17:57:42.375496+0000","last_became_active":"2024-03-16T17:57:41.402696+0000","last_became_peered":"2024-03-16T17:57:41.402696+0000","last_unstale":"2024-03-16T17:57:42.375496+0000","last_undegraded":"2024-03-16T17:57:42.375496+0000","last_fullsized":"2024-03-16T17:57:42.375496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:16:17.454713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733543+0000","last_change":"2024-03-16T17:57:40.378607+0000","last_active":"2024-03-16T17:57:42.733543+0000","last_peered":"2024-03-16T17:57:42.733543+0000","last_clean":"2024-03-16T17:57:42.733543+0000","last_became_active":"2024-03-16T17:57:40.378291+0000","last_became_peered":"2024-03-16T17:57:40.378291+0000","last_unstale":"2024-03-16T17:57:42.733543+0000","last_undegraded":"2024-03-16T17:57:42.733543+0000","last_fullsized":"2024-03-16T17:57:42.733543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:16.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735929+0000","last_change":"2024-03-16T17:57:41.409430+0000","last_active":"2024-03-16T17:57:42.735929+0000","last_peered":"2024-03-16T17:57:42.735929+0000","last_clean":"2024-03-16T17:57:42.735929+0000","last_became_active":"2024-03-16T17:57:41.409209+0000","last_became_peered":"2024-03-16T17:57:41.409209+0000","last_unstale":"2024-03-16T17:57:42.735929+0000","last_undegraded":"2024-03-16T17:57:42.735929+0000","last_fullsized":"2024-03-16T17:57:42.735929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:58.061341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733228+0000","last_change":"2024-03-16T17:57:35.560172+0000","last_active":"2024-03-16T17:57:42.733228+0000","last_peered":"2024-03-16T17:57:42.733228+0000","last_clean":"2024-03-16T17:57:42.733228+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T17:57:42.733228+0000","last_undegraded":"2024-03-16T17:57:42.733228+0000","last_fullsized":"2024-03-16T17:57:42.733228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_clean_scrub_stamp":"2024-03-16T17:57:34.174437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:20:03.039404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374626+0000","last_change":"2024-03-16T17:57:40.372426+0000","last_active":"2024-03-16T17:57:42.374626+0000","last_peered":"2024-03-16T17:57:42.374626+0000","last_clean":"2024-03-16T17:57:42.374626+0000","last_became_active":"2024-03-16T17:57:40.372097+0000","last_became_peered":"2024-03-16T17:57:40.372097+0000","last_unstale":"2024-03-16T17:57:42.374626+0000","last_undegraded":"2024-03-16T17:57:42.374626+0000","last_fullsized":"2024-03-16T17:57:42.374626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734031+0000","last_change":"2024-03-16T17:57:41.401095+0000","last_active":"2024-03-16T17:57:42.734031+0000","last_peered":"2024-03-16T17:57:42.734031+0000","last_clean":"2024-03-16T17:57:42.734031+0000","last_became_active":"2024-03-16T17:57:41.400788+0000","last_became_peered":"2024-03-16T17:57:41.400788+0000","last_unstale":"2024-03-16T17:57:42.734031+0000","last_undegraded":"2024-03-16T17:57:42.734031+0000","last_fullsized":"2024-03-16T17:57:42.734031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733259+0000","last_change":"2024-03-16T17:57:40.369748+0000","last_active":"2024-03-16T17:57:42.733259+0000","last_peered":"2024-03-16T17:57:42.733259+0000","last_clean":"2024-03-16T17:57:42.733259+0000","last_became_active":"2024-03-16T17:57:40.368850+0000","last_became_peered":"2024-03-16T17:57:40.368850+0000","last_unstale":"2024-03-16T17:57:42.733259+0000","last_undegraded":"2024-03-16T17:57:42.733259+0000","last_fullsized":"2024-03-16T17:57:42.733259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:31:16.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732283+0000","last_change":"2024-03-16T17:57:41.373872+0000","last_active":"2024-03-16T17:57:42.732283+0000","last_peered":"2024-03-16T17:57:42.732283+0000","last_clean":"2024-03-16T17:57:42.732283+0000","last_became_active":"2024-03-16T17:57:41.373478+0000","last_became_peered":"2024-03-16T17:57:41.373478+0000","last_unstale":"2024-03-16T17:57:42.732283+0000","last_undegraded":"2024-03-16T17:57:42.732283+0000","last_fullsized":"2024-03-16T17:57:42.732283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:04:23.189434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733503+0000","last_change":"2024-03-16T17:57:40.373316+0000","last_active":"2024-03-16T17:57:42.733503+0000","last_peered":"2024-03-16T17:57:42.733503+0000","last_clean":"2024-03-16T17:57:42.733503+0000","last_became_active":"2024-03-16T17:57:40.373015+0000","last_became_peered":"2024-03-16T17:57:40.373015+0000","last_unstale":"2024-03-16T17:57:42.733503+0000","last_undegraded":"2024-03-16T17:57:42.733503+0000","last_fullsized":"2024-03-16T17:57:42.733503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:42:38.164765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369425+0000","last_change":"2024-03-16T17:57:41.383198+0000","last_active":"2024-03-16T17:57:42.369425+0000","last_peered":"2024-03-16T17:57:42.369425+0000","last_clean":"2024-03-16T17:57:42.369425+0000","last_became_active":"2024-03-16T17:57:41.382886+0000","last_became_peered":"2024-03-16T17:57:41.382886+0000","last_unstale":"2024-03-16T17:57:42.369425+0000","last_undegraded":"2024-03-16T17:57:42.369425+0000","last_fullsized":"2024-03-16T17:57:42.369425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375951+0000","last_change":"2024-03-16T17:57:40.368934+0000","last_active":"2024-03-16T17:57:42.375951+0000","last_peered":"2024-03-16T17:57:42.375951+0000","last_clean":"2024-03-16T17:57:42.375951+0000","last_became_active":"2024-03-16T17:57:40.368496+0000","last_became_peered":"2024-03-16T17:57:40.368496+0000","last_unstale":"2024-03-16T17:57:42.375951+0000","last_undegraded":"2024-03-16T17:57:42.375951+0000","last_fullsized":"2024-03-16T17:57:42.375951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373158+0000","last_change":"2024-03-16T17:57:41.377235+0000","last_active":"2024-03-16T17:57:42.373158+0000","last_peered":"2024-03-16T17:57:42.373158+0000","last_clean":"2024-03-16T17:57:42.373158+0000","last_became_active":"2024-03-16T17:57:41.376796+0000","last_became_peered":"2024-03-16T17:57:41.376796+0000","last_unstale":"2024-03-16T17:57:42.373158+0000","last_undegraded":"2024-03-16T17:57:42.373158+0000","last_fullsized":"2024-03-16T17:57:42.373158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:24:23.606350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369894+0000","last_change":"2024-03-16T17:57:40.365747+0000","last_active":"2024-03-16T17:57:42.369894+0000","last_peered":"2024-03-16T17:57:42.369894+0000","last_clean":"2024-03-16T17:57:42.369894+0000","last_became_active":"2024-03-16T17:57:40.364579+0000","last_became_peered":"2024-03-16T17:57:40.364579+0000","last_unstale":"2024-03-16T17:57:42.369894+0000","last_undegraded":"2024-03-16T17:57:42.369894+0000","last_fullsized":"2024-03-16T17:57:42.369894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376270+0000","last_change":"2024-03-16T17:57:41.404470+0000","last_active":"2024-03-16T17:57:42.376270+0000","last_peered":"2024-03-16T17:57:42.376270+0000","last_clean":"2024-03-16T17:57:42.376270+0000","last_became_active":"2024-03-16T17:57:41.404100+0000","last_became_peered":"2024-03-16T17:57:41.404100+0000","last_unstale":"2024-03-16T17:57:42.376270+0000","last_undegraded":"2024-03-16T17:57:42.376270+0000","last_fullsized":"2024-03-16T17:57:42.376270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:50:09.742348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736227+0000","last_change":"2024-03-16T17:57:40.370118+0000","last_active":"2024-03-16T17:57:42.736227+0000","last_peered":"2024-03-16T17:57:42.736227+0000","last_clean":"2024-03-16T17:57:42.736227+0000","last_became_active":"2024-03-16T17:57:40.369751+0000","last_became_peered":"2024-03-16T17:57:40.369751+0000","last_unstale":"2024-03-16T17:57:42.736227+0000","last_undegraded":"2024-03-16T17:57:42.736227+0000","last_fullsized":"2024-03-16T17:57:42.736227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:04:22.171852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369793+0000","last_change":"2024-03-16T17:57:41.384659+0000","last_active":"2024-03-16T17:57:42.369793+0000","last_peered":"2024-03-16T17:57:42.369793+0000","last_clean":"2024-03-16T17:57:42.369793+0000","last_became_active":"2024-03-16T17:57:41.384374+0000","last_became_peered":"2024-03-16T17:57:41.384374+0000","last_unstale":"2024-03-16T17:57:42.369793+0000","last_undegraded":"2024-03-16T17:57:42.369793+0000","last_fullsized":"2024-03-16T17:57:42.369793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370123+0000","last_change":"2024-03-16T17:57:40.377982+0000","last_active":"2024-03-16T17:57:42.370123+0000","last_peered":"2024-03-16T17:57:42.370123+0000","last_clean":"2024-03-16T17:57:42.370123+0000","last_became_active":"2024-03-16T17:57:40.377669+0000","last_became_peered":"2024-03-16T17:57:40.377669+0000","last_unstale":"2024-03-16T17:57:42.370123+0000","last_undegraded":"2024-03-16T17:57:42.370123+0000","last_fullsized":"2024-03-16T17:57:42.370123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:31:16.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735021+0000","last_change":"2024-03-16T17:57:41.400625+0000","last_active":"2024-03-16T17:57:42.735021+0000","last_peered":"2024-03-16T17:57:42.735021+0000","last_clean":"2024-03-16T17:57:42.735021+0000","last_became_active":"2024-03-16T17:57:41.398361+0000","last_became_peered":"2024-03-16T17:57:41.398361+0000","last_unstale":"2024-03-16T17:57:42.735021+0000","last_undegraded":"2024-03-16T17:57:42.735021+0000","last_fullsized":"2024-03-16T17:57:42.735021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733837+0000","last_change":"2024-03-16T17:57:40.379426+0000","last_active":"2024-03-16T17:57:42.733837+0000","last_peered":"2024-03-16T17:57:42.733837+0000","last_clean":"2024-03-16T17:57:42.733837+0000","last_became_active":"2024-03-16T17:57:40.379136+0000","last_became_peered":"2024-03-16T17:57:40.379136+0000","last_unstale":"2024-03-16T17:57:42.733837+0000","last_undegraded":"2024-03-16T17:57:42.733837+0000","last_fullsized":"2024-03-16T17:57:42.733837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732957+0000","last_change":"2024-03-16T17:57:41.379286+0000","last_active":"2024-03-16T17:57:42.732957+0000","last_peered":"2024-03-16T17:57:42.732957+0000","last_clean":"2024-03-16T17:57:42.732957+0000","last_became_active":"2024-03-16T17:57:41.378998+0000","last_became_peered":"2024-03-16T17:57:41.378998+0000","last_unstale":"2024-03-16T17:57:42.732957+0000","last_undegraded":"2024-03-16T17:57:42.732957+0000","last_fullsized":"2024-03-16T17:57:42.732957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732976+0000","last_change":"2024-03-16T17:57:40.369141+0000","last_active":"2024-03-16T17:57:42.732976+0000","last_peered":"2024-03-16T17:57:42.732976+0000","last_clean":"2024-03-16T17:57:42.732976+0000","last_became_active":"2024-03-16T17:57:40.368542+0000","last_became_peered":"2024-03-16T17:57:40.368542+0000","last_unstale":"2024-03-16T17:57:42.732976+0000","last_undegraded":"2024-03-16T17:57:42.732976+0000","last_fullsized":"2024-03-16T17:57:42.732976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.372617+0000","last_change":"2024-03-16T17:57:41.375830+0000","last_active":"2024-03-16T17:57:42.372617+0000","last_peered":"2024-03-16T17:57:42.372617+0000","last_clean":"2024-03-16T17:57:42.372617+0000","last_became_active":"2024-03-16T17:57:41.375515+0000","last_became_peered":"2024-03-16T17:57:41.375515+0000","last_unstale":"2024-03-16T17:57:42.372617+0000","last_undegraded":"2024-03-16T17:57:42.372617+0000","last_fullsized":"2024-03-16T17:57:42.372617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:32:25.075362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369244+0000","last_change":"2024-03-16T17:57:40.367142+0000","last_active":"2024-03-16T17:57:42.369244+0000","last_peered":"2024-03-16T17:57:42.369244+0000","last_clean":"2024-03-16T17:57:42.369244+0000","last_became_active":"2024-03-16T17:57:40.366778+0000","last_became_peered":"2024-03-16T17:57:40.366778+0000","last_unstale":"2024-03-16T17:57:42.369244+0000","last_undegraded":"2024-03-16T17:57:42.369244+0000","last_fullsized":"2024-03-16T17:57:42.369244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732297+0000","last_change":"2024-03-16T17:57:41.388886+0000","last_active":"2024-03-16T17:57:42.732297+0000","last_peered":"2024-03-16T17:57:42.732297+0000","last_clean":"2024-03-16T17:57:42.732297+0000","last_became_active":"2024-03-16T17:57:41.388588+0000","last_became_peered":"2024-03-16T17:57:41.388588+0000","last_unstale":"2024-03-16T17:57:42.732297+0000","last_undegraded":"2024-03-16T17:57:42.732297+0000","last_fullsized":"2024-03-16T17:57:42.732297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369599+0000","last_change":"2024-03-16T17:57:40.371549+0000","last_active":"2024-03-16T17:57:42.369599+0000","last_peered":"2024-03-16T17:57:42.369599+0000","last_clean":"2024-03-16T17:57:42.369599+0000","last_became_active":"2024-03-16T17:57:40.371257+0000","last_became_peered":"2024-03-16T17:57:40.371257+0000","last_unstale":"2024-03-16T17:57:42.369599+0000","last_undegraded":"2024-03-16T17:57:42.369599+0000","last_fullsized":"2024-03-16T17:57:42.369599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734385+0000","last_change":"2024-03-16T17:57:41.392961+0000","last_active":"2024-03-16T17:57:42.734385+0000","last_peered":"2024-03-16T17:57:42.734385+0000","last_clean":"2024-03-16T17:57:42.734385+0000","last_became_active":"2024-03-16T17:57:41.392665+0000","last_became_peered":"2024-03-16T17:57:41.392665+0000","last_unstale":"2024-03-16T17:57:42.734385+0000","last_undegraded":"2024-03-16T17:57:42.734385+0000","last_fullsized":"2024-03-16T17:57:42.734385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735443+0000","last_change":"2024-03-16T17:57:40.376135+0000","last_active":"2024-03-16T17:57:42.735443+0000","last_peered":"2024-03-16T17:57:42.735443+0000","last_clean":"2024-03-16T17:57:42.735443+0000","last_became_active":"2024-03-16T17:57:40.375292+0000","last_became_peered":"2024-03-16T17:57:40.375292+0000","last_unstale":"2024-03-16T17:57:42.735443+0000","last_undegraded":"2024-03-16T17:57:42.735443+0000","last_fullsized":"2024-03-16T17:57:42.735443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:16.023603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733288+0000","last_change":"2024-03-16T17:57:41.394268+0000","last_active":"2024-03-16T17:57:42.733288+0000","last_peered":"2024-03-16T17:57:42.733288+0000","last_clean":"2024-03-16T17:57:42.733288+0000","last_became_active":"2024-03-16T17:57:41.393941+0000","last_became_peered":"2024-03-16T17:57:41.393941+0000","last_unstale":"2024-03-16T17:57:42.733288+0000","last_undegraded":"2024-03-16T17:57:42.733288+0000","last_fullsized":"2024-03-16T17:57:42.733288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376119+0000","last_change":"2024-03-16T17:57:41.402175+0000","last_active":"2024-03-16T17:57:42.376119+0000","last_peered":"2024-03-16T17:57:42.376119+0000","last_clean":"2024-03-16T17:57:42.376119+0000","last_became_active":"2024-03-16T17:57:41.401851+0000","last_became_peered":"2024-03-16T17:57:41.401851+0000","last_unstale":"2024-03-16T17:57:42.376119+0000","last_undegraded":"2024-03-16T17:57:42.376119+0000","last_fullsized":"2024-03-16T17:57:42.376119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:58.061341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734977+0000","last_change":"2024-03-16T17:57:40.377332+0000","last_active":"2024-03-16T17:57:42.734977+0000","last_peered":"2024-03-16T17:57:42.734977+0000","last_clean":"2024-03-16T17:57:42.734977+0000","last_became_active":"2024-03-16T17:57:40.377028+0000","last_became_peered":"2024-03-16T17:57:40.377028+0000","last_unstale":"2024-03-16T17:57:42.734977+0000","last_undegraded":"2024-03-16T17:57:42.734977+0000","last_fullsized":"2024-03-16T17:57:42.734977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375244+0000","last_change":"2024-03-16T17:57:41.402689+0000","last_active":"2024-03-16T17:57:42.375244+0000","last_peered":"2024-03-16T17:57:42.375244+0000","last_clean":"2024-03-16T17:57:42.375244+0000","last_became_active":"2024-03-16T17:57:41.401253+0000","last_became_peered":"2024-03-16T17:57:41.401253+0000","last_unstale":"2024-03-16T17:57:42.375244+0000","last_undegraded":"2024-03-16T17:57:42.375244+0000","last_fullsized":"2024-03-16T17:57:42.375244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735773+0000","last_change":"2024-03-16T17:57:40.379396+0000","last_active":"2024-03-16T17:57:42.735773+0000","last_peered":"2024-03-16T17:57:42.735773+0000","last_clean":"2024-03-16T17:57:42.735773+0000","last_became_active":"2024-03-16T17:57:40.379058+0000","last_became_peered":"2024-03-16T17:57:40.379058+0000","last_unstale":"2024-03-16T17:57:42.735773+0000","last_undegraded":"2024-03-16T17:57:42.735773+0000","last_fullsized":"2024-03-16T17:57:42.735773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:53:59.451003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732999+0000","last_change":"2024-03-16T17:57:41.407792+0000","last_active":"2024-03-16T17:57:42.732999+0000","last_peered":"2024-03-16T17:57:42.732999+0000","last_clean":"2024-03-16T17:57:42.732999+0000","last_became_active":"2024-03-16T17:57:41.407412+0000","last_became_peered":"2024-03-16T17:57:41.407412+0000","last_unstale":"2024-03-16T17:57:42.732999+0000","last_undegraded":"2024-03-16T17:57:42.732999+0000","last_fullsized":"2024-03-16T17:57:42.732999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735117+0000","last_change":"2024-03-16T17:57:40.366569+0000","last_active":"2024-03-16T17:57:42.735117+0000","last_peered":"2024-03-16T17:57:42.735117+0000","last_clean":"2024-03-16T17:57:42.735117+0000","last_became_active":"2024-03-16T17:57:40.362328+0000","last_became_peered":"2024-03-16T17:57:40.362328+0000","last_unstale":"2024-03-16T17:57:42.735117+0000","last_undegraded":"2024-03-16T17:57:42.735117+0000","last_fullsized":"2024-03-16T17:57:42.735117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734985+0000","last_change":"2024-03-16T17:57:41.400212+0000","last_active":"2024-03-16T17:57:42.734985+0000","last_peered":"2024-03-16T17:57:42.734985+0000","last_clean":"2024-03-16T17:57:42.734985+0000","last_became_active":"2024-03-16T17:57:41.399937+0000","last_became_peered":"2024-03-16T17:57:41.399937+0000","last_unstale":"2024-03-16T17:57:42.734985+0000","last_undegraded":"2024-03-16T17:57:42.734985+0000","last_fullsized":"2024-03-16T17:57:42.734985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733309+0000","last_change":"2024-03-16T17:57:40.368635+0000","last_active":"2024-03-16T17:57:42.733309+0000","last_peered":"2024-03-16T17:57:42.733309+0000","last_clean":"2024-03-16T17:57:42.733309+0000","last_became_active":"2024-03-16T17:57:40.368370+0000","last_became_peered":"2024-03-16T17:57:40.368370+0000","last_unstale":"2024-03-16T17:57:42.733309+0000","last_undegraded":"2024-03-16T17:57:42.733309+0000","last_fullsized":"2024-03-16T17:57:42.733309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:19.918231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":658813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":69475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":67953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":69475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T17:57:45.757 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T17:57:45.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.985+0000 7f87b5800640 1 Processor -- start 2024-03-16T17:57:45.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.985+0000 7f87b5800640 1 -- start start 2024-03-16T17:57:45.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.987+0000 7f87b5800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b00d0040 0x7f87b0166c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.987+0000 7f87b5800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b00b4b90 con 0x7f87b01454d0 2024-03-16T17:57:45.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.987+0000 7f87b5800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b00cf500 con 0x7f87b00cdf80 2024-03-16T17:57:45.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.987+0000 7f87b5800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b01458b0 con 0x7f87b00d0040 2024-03-16T17:57:45.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.987+0000 7f87aea00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f87b01454d0 0x7f87b00b34c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56156/0 (socket says 172.21.15.82:56156) 2024-03-16T17:57:45.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.987+0000 7f87aea00640 1 -- 172.21.15.82:0/4014384111 learned_addr learned my addr 172.21.15.82:0/4014384111 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.987+0000 7f87af400640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b00d0040 0x7f87b0166c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.987+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461144440 0 0) 0x7f87b00b4b90 con 0x7f87b01454d0 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.987+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8794003520 con 0x7f87b01454d0 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2312804432 0 0) 0x7f8794003520 con 0x7f87b01454d0 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b00d0040 msgr2=0x7f87b0166c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b00d0040 0x7f87b0166c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 >> v1:172.21.15.90:6789/0 conn(0x7f87b00cdf80 legacy=0x7f87b00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b00ce460 con 0x7f87b01454d0 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f87a0001670 con 0x7f87b01454d0 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 >> v1:172.21.15.82:6789/0 conn(0x7f87b01454d0 legacy=0x7f87b00b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 0x7f8794003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87940047c0 0x7f8794004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b00d0040 0x7f8794005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b01458b0 con 0x7f87940039a0 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879400ba60 con 0x7f87940047c0 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879400bc20 con 0x7f87b00d0040 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87aea00640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 0x7f8794003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87ae000640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87940047c0 0x7f8794004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.988+0000 7f87af400640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b00d0040 0x7f8794005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.989+0000 7f87aea00640 1 -- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b00d0040 msgr2=0x7f8794005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.989+0000 7f87aea00640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b00d0040 0x7f8794005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.989+0000 7f87aea00640 1 -- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87940047c0 msgr2=0x7f8794004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.989+0000 7f87aea00640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87940047c0 0x7f8794004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.989+0000 7f87aea00640 1 -- 172.21.15.82:0/4014384111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8794003520 con 0x7f87940039a0 2024-03-16T17:57:45.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.989+0000 7f87af400640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b00d0040 0x7f8794005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:45.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.990+0000 7f87aea00640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 0x7f8794003dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c210 tx=0x7f87940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2a220ac799d8c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:45.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.990+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0005ad0 con 0x7f87940039a0 2024-03-16T17:57:45.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.990+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a0005c30 con 0x7f87940039a0 2024-03-16T17:57:45.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.990+0000 7f87ad600640 1 -- 172.21.15.82:0/4014384111 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a002c870 con 0x7f87940039a0 2024-03-16T17:57:45.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.991+0000 7f87b5800640 1 -- 172.21.15.82:0/4014384111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 msgr2=0x7f8794003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.991+0000 7f87b5800640 1 --2- 172.21.15.82:0/4014384111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 0x7f8794003dc0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c210 tx=0x7f87940057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.991+0000 7f87af400640 1 -- 172.21.15.82:0/4014384111 reap_dead start 2024-03-16T17:57:45.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.991+0000 7f87b5800640 1 -- 172.21.15.82:0/4014384111 shutdown_connections 2024-03-16T17:57:45.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.991+0000 7f87b5800640 1 -- 172.21.15.82:0/4014384111 >> 172.21.15.82:0/4014384111 conn(0x7f87b006ae70 msgr2=0x7f87b006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:45.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.991+0000 7f87b5800640 1 -- 172.21.15.82:0/4014384111 shutdown_connections 2024-03-16T17:57:45.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.991+0000 7f87b5800640 1 -- 172.21.15.82:0/4014384111 wait complete. 2024-03-16T17:57:45.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.992+0000 7f87b5800640 1 Processor -- start 2024-03-16T17:57:45.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.993+0000 7f87b5800640 1 -- start start 2024-03-16T17:57:45.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.993+0000 7f87b5800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 0x7f87b015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.993+0000 7f87b5800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b015a3e0 0x7f87b01f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.993+0000 7f87b5800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b01f32f0 0x7f87b0132b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87b5800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00aab50 con 0x7f87940039a0 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87b5800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b01f46e0 con 0x7f87b015a3e0 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87b5800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b015a8a0 con 0x7f87b01f32f0 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87aea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 0x7f87b015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87aea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 0x7f87b015b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35126/0 (socket says 172.21.15.82:35126) 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87aea00640 1 -- 172.21.15.82:0/633762137 learned_addr learned my addr 172.21.15.82:0/633762137 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87af400640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b01f32f0 0x7f87b0132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87ae000640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b015a3e0 0x7f87b01f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87aea00640 1 -- 172.21.15.82:0/633762137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b01f32f0 msgr2=0x7f87b0132b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87aea00640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b01f32f0 0x7f87b0132b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87aea00640 1 -- 172.21.15.82:0/633762137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b015a3e0 msgr2=0x7f87b01f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87aea00640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b015a3e0 0x7f87b01f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:45.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87aea00640 1 -- 172.21.15.82:0/633762137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b015ab80 con 0x7f87940039a0 2024-03-16T17:57:46.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87af400640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b01f32f0 0x7f87b0132b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:46.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87ae000640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b015a3e0 0x7f87b01f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:46.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.994+0000 7f87aea00640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 0x7f87b015b940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c210 tx=0x7f87a0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:46.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.995+0000 7f87a6a00640 1 -- 172.21.15.82:0/633762137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a002cc80 con 0x7f87940039a0 2024-03-16T17:57:46.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.995+0000 7f87a6a00640 1 -- 172.21.15.82:0/633762137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a002ce00 con 0x7f87940039a0 2024-03-16T17:57:46.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.995+0000 7f87a6a00640 1 -- 172.21.15.82:0/633762137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a00072b0 con 0x7f87940039a0 2024-03-16T17:57:46.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.995+0000 7f87b5800640 1 -- 172.21.15.82:0/633762137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b015ae60 con 0x7f87940039a0 2024-03-16T17:57:46.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.995+0000 7f87b5800640 1 -- 172.21.15.82:0/633762137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b015b420 con 0x7f87940039a0 2024-03-16T17:57:46.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.996+0000 7f87b5800640 1 -- 172.21.15.82:0/633762137 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b015b5b0 con 0x7f87940039a0 2024-03-16T17:57:46.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.997+0000 7f87a6a00640 1 -- 172.21.15.82:0/633762137 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f87a0032070 con 0x7f87940039a0 2024-03-16T17:57:46.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.997+0000 7f87a6a00640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f879006b3b0 0x7f879006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.998+0000 7f87a6a00640 1 -- 172.21.15.82:0/633762137 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87a00aaf00 con 0x7f87940039a0 2024-03-16T17:57:46.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.998+0000 7f87ae000640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f879006b3b0 0x7f879006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:45.998+0000 7f87ae000640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f879006b3b0 0x7f879006d870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87b015c0f0 tx=0x7f879c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:46.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.002+0000 7f87a6a00640 1 -- 172.21.15.82:0/633762137 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f87a0075b60 con 0x7f87940039a0 2024-03-16T17:57:46.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.195+0000 7f87b5800640 1 -- 172.21.15.82:0/633762137 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87b00cea20 con 0x7f879006b3b0 2024-03-16T17:57:46.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.208+0000 7f87a6a00640 1 -- 172.21.15.82:0/633762137 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251947 (secure 0 0 0) 0x7f879c001490 con 0x7f879006b3b0 2024-03-16T17:57:46.214 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:46.216 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 -- 172.21.15.82:0/633762137 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f879006b3b0 msgr2=0x7f879006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f879006b3b0 0x7f879006d870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f87b015c0f0 tx=0x7f879c009040 comp rx=0 tx=0).stop 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 -- 172.21.15.82:0/633762137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 msgr2=0x7f87b015b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 0x7f87b015b940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c210 tx=0x7f87a0001a00 comp rx=0 tx=0).stop 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 -- 172.21.15.82:0/633762137 shutdown_connections 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f879006b3b0 0x7f879006d870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f87b01f32f0 0x7f87b0132b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87b015a3e0 0x7f87b01f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 --2- 172.21.15.82:0/633762137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87940039a0 0x7f87b015b940 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 -- 172.21.15.82:0/633762137 >> 172.21.15.82:0/633762137 conn(0x7f87b006ae70 msgr2=0x7f87b0143c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 -- 172.21.15.82:0/633762137 shutdown_connections 2024-03-16T17:57:46.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.214+0000 7f87b5800640 1 -- 172.21.15.82:0/633762137 wait complete. 2024-03-16T17:57:46.235 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-16T17:57:46.011936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":140,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219036,"kb_used_data":2840,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755684,"statfs":{"total":773094113280,"available":772869820416,"internally_reserved":0,"allocated":2908160,"data_stored":1753660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001986"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736016+0000","last_change":"2024-03-16T17:57:41.412158+0000","last_active":"2024-03-16T17:57:42.736016+0000","last_peered":"2024-03-16T17:57:42.736016+0000","last_clean":"2024-03-16T17:57:42.736016+0000","last_became_active":"2024-03-16T17:57:41.411766+0000","last_became_peered":"2024-03-16T17:57:41.411766+0000","last_unstale":"2024-03-16T17:57:42.736016+0000","last_undegraded":"2024-03-16T17:57:42.736016+0000","last_fullsized":"2024-03-16T17:57:42.736016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:50:09.742348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370170+0000","last_change":"2024-03-16T17:57:41.383449+0000","last_active":"2024-03-16T17:57:42.370170+0000","last_peered":"2024-03-16T17:57:42.370170+0000","last_clean":"2024-03-16T17:57:42.370170+0000","last_became_active":"2024-03-16T17:57:41.382165+0000","last_became_peered":"2024-03-16T17:57:41.382165+0000","last_unstale":"2024-03-16T17:57:42.370170+0000","last_undegraded":"2024-03-16T17:57:42.370170+0000","last_fullsized":"2024-03-16T17:57:42.370170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732651+0000","last_change":"2024-03-16T17:57:41.380082+0000","last_active":"2024-03-16T17:57:42.732651+0000","last_peered":"2024-03-16T17:57:42.732651+0000","last_clean":"2024-03-16T17:57:42.732651+0000","last_became_active":"2024-03-16T17:57:41.379508+0000","last_became_peered":"2024-03-16T17:57:41.379508+0000","last_unstale":"2024-03-16T17:57:42.732651+0000","last_undegraded":"2024-03-16T17:57:42.732651+0000","last_fullsized":"2024-03-16T17:57:42.732651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375083+0000","last_change":"2024-03-16T17:57:41.404364+0000","last_active":"2024-03-16T17:57:42.375083+0000","last_peered":"2024-03-16T17:57:42.375083+0000","last_clean":"2024-03-16T17:57:42.375083+0000","last_became_active":"2024-03-16T17:57:41.404048+0000","last_became_peered":"2024-03-16T17:57:41.404048+0000","last_unstale":"2024-03-16T17:57:42.375083+0000","last_undegraded":"2024-03-16T17:57:42.375083+0000","last_fullsized":"2024-03-16T17:57:42.375083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734821+0000","last_change":"2024-03-16T17:57:41.392327+0000","last_active":"2024-03-16T17:57:42.734821+0000","last_peered":"2024-03-16T17:57:42.734821+0000","last_clean":"2024-03-16T17:57:42.734821+0000","last_became_active":"2024-03-16T17:57:41.392007+0000","last_became_peered":"2024-03-16T17:57:41.392007+0000","last_unstale":"2024-03-16T17:57:42.734821+0000","last_undegraded":"2024-03-16T17:57:42.734821+0000","last_fullsized":"2024-03-16T17:57:42.734821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:48:10.857666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375324+0000","last_change":"2024-03-16T17:57:41.398446+0000","last_active":"2024-03-16T17:57:42.375324+0000","last_peered":"2024-03-16T17:57:42.375324+0000","last_clean":"2024-03-16T17:57:42.375324+0000","last_became_active":"2024-03-16T17:57:41.398135+0000","last_became_peered":"2024-03-16T17:57:41.398135+0000","last_unstale":"2024-03-16T17:57:42.375324+0000","last_undegraded":"2024-03-16T17:57:42.375324+0000","last_fullsized":"2024-03-16T17:57:42.375324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:48:10.857666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734440+0000","last_change":"2024-03-16T17:57:41.405535+0000","last_active":"2024-03-16T17:57:42.734440+0000","last_peered":"2024-03-16T17:57:42.734440+0000","last_clean":"2024-03-16T17:57:42.734440+0000","last_became_active":"2024-03-16T17:57:41.405323+0000","last_became_peered":"2024-03-16T17:57:41.405323+0000","last_unstale":"2024-03-16T17:57:42.734440+0000","last_undegraded":"2024-03-16T17:57:42.734440+0000","last_fullsized":"2024-03-16T17:57:42.734440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.731940+0000","last_change":"2024-03-16T17:57:41.402459+0000","last_active":"2024-03-16T17:57:42.731940+0000","last_peered":"2024-03-16T17:57:42.731940+0000","last_clean":"2024-03-16T17:57:42.731940+0000","last_became_active":"2024-03-16T17:57:41.402103+0000","last_became_peered":"2024-03-16T17:57:41.402103+0000","last_unstale":"2024-03-16T17:57:42.731940+0000","last_undegraded":"2024-03-16T17:57:42.731940+0000","last_fullsized":"2024-03-16T17:57:42.731940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:20.628509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736292+0000","last_change":"2024-03-16T17:57:41.409011+0000","last_active":"2024-03-16T17:57:42.736292+0000","last_peered":"2024-03-16T17:57:42.736292+0000","last_clean":"2024-03-16T17:57:42.736292+0000","last_became_active":"2024-03-16T17:57:41.408811+0000","last_became_peered":"2024-03-16T17:57:41.408811+0000","last_unstale":"2024-03-16T17:57:42.736292+0000","last_undegraded":"2024-03-16T17:57:42.736292+0000","last_fullsized":"2024-03-16T17:57:42.736292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:11:48.511692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734591+0000","last_change":"2024-03-16T17:57:41.401707+0000","last_active":"2024-03-16T17:57:42.734591+0000","last_peered":"2024-03-16T17:57:42.734591+0000","last_clean":"2024-03-16T17:57:42.734591+0000","last_became_active":"2024-03-16T17:57:41.401330+0000","last_became_peered":"2024-03-16T17:57:41.401330+0000","last_unstale":"2024-03-16T17:57:42.734591+0000","last_undegraded":"2024-03-16T17:57:42.734591+0000","last_fullsized":"2024-03-16T17:57:42.734591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733619+0000","last_change":"2024-03-16T17:57:41.399956+0000","last_active":"2024-03-16T17:57:42.733619+0000","last_peered":"2024-03-16T17:57:42.733619+0000","last_clean":"2024-03-16T17:57:42.733619+0000","last_became_active":"2024-03-16T17:57:41.399696+0000","last_became_peered":"2024-03-16T17:57:41.399696+0000","last_unstale":"2024-03-16T17:57:42.733619+0000","last_undegraded":"2024-03-16T17:57:42.733619+0000","last_fullsized":"2024-03-16T17:57:42.733619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373739+0000","last_change":"2024-03-16T17:57:41.397224+0000","last_active":"2024-03-16T17:57:42.373739+0000","last_peered":"2024-03-16T17:57:42.373739+0000","last_clean":"2024-03-16T17:57:42.373739+0000","last_became_active":"2024-03-16T17:57:41.396857+0000","last_became_peered":"2024-03-16T17:57:41.396857+0000","last_unstale":"2024-03-16T17:57:42.373739+0000","last_undegraded":"2024-03-16T17:57:42.373739+0000","last_fullsized":"2024-03-16T17:57:42.373739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:32:52.837630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734251+0000","last_change":"2024-03-16T17:57:41.399774+0000","last_active":"2024-03-16T17:57:42.734251+0000","last_peered":"2024-03-16T17:57:42.734251+0000","last_clean":"2024-03-16T17:57:42.734251+0000","last_became_active":"2024-03-16T17:57:41.399505+0000","last_became_peered":"2024-03-16T17:57:41.399505+0000","last_unstale":"2024-03-16T17:57:42.734251+0000","last_undegraded":"2024-03-16T17:57:42.734251+0000","last_fullsized":"2024-03-16T17:57:42.734251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732264+0000","last_change":"2024-03-16T17:57:41.380132+0000","last_active":"2024-03-16T17:57:42.732264+0000","last_peered":"2024-03-16T17:57:42.732264+0000","last_clean":"2024-03-16T17:57:42.732264+0000","last_became_active":"2024-03-16T17:57:41.379536+0000","last_became_peered":"2024-03-16T17:57:41.379536+0000","last_unstale":"2024-03-16T17:57:42.732264+0000","last_undegraded":"2024-03-16T17:57:42.732264+0000","last_fullsized":"2024-03-16T17:57:42.732264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:54:00.468586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369395+0000","last_change":"2024-03-16T17:57:41.393322+0000","last_active":"2024-03-16T17:57:42.369395+0000","last_peered":"2024-03-16T17:57:42.369395+0000","last_clean":"2024-03-16T17:57:42.369395+0000","last_became_active":"2024-03-16T17:57:41.391272+0000","last_became_peered":"2024-03-16T17:57:41.391272+0000","last_unstale":"2024-03-16T17:57:42.369395+0000","last_undegraded":"2024-03-16T17:57:42.369395+0000","last_fullsized":"2024-03-16T17:57:42.369395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374104+0000","last_change":"2024-03-16T17:57:41.399992+0000","last_active":"2024-03-16T17:57:42.374104+0000","last_peered":"2024-03-16T17:57:42.374104+0000","last_clean":"2024-03-16T17:57:42.374104+0000","last_became_active":"2024-03-16T17:57:41.399406+0000","last_became_peered":"2024-03-16T17:57:41.399406+0000","last_unstale":"2024-03-16T17:57:42.374104+0000","last_undegraded":"2024-03-16T17:57:42.374104+0000","last_fullsized":"2024-03-16T17:57:42.374104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376607+0000","last_change":"2024-03-16T17:57:41.416986+0000","last_active":"2024-03-16T17:57:42.376607+0000","last_peered":"2024-03-16T17:57:42.376607+0000","last_clean":"2024-03-16T17:57:42.376607+0000","last_became_active":"2024-03-16T17:57:41.416699+0000","last_became_peered":"2024-03-16T17:57:41.416699+0000","last_unstale":"2024-03-16T17:57:42.376607+0000","last_undegraded":"2024-03-16T17:57:42.376607+0000","last_fullsized":"2024-03-16T17:57:42.376607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:11:48.511692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733913+0000","last_change":"2024-03-16T17:57:41.394311+0000","last_active":"2024-03-16T17:57:42.733913+0000","last_peered":"2024-03-16T17:57:42.733913+0000","last_clean":"2024-03-16T17:57:42.733913+0000","last_became_active":"2024-03-16T17:57:41.393750+0000","last_became_peered":"2024-03-16T17:57:41.393750+0000","last_unstale":"2024-03-16T17:57:42.733913+0000","last_undegraded":"2024-03-16T17:57:42.733913+0000","last_fullsized":"2024-03-16T17:57:42.733913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:55:54.818895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736015+0000","last_change":"2024-03-16T17:57:41.408696+0000","last_active":"2024-03-16T17:57:42.736015+0000","last_peered":"2024-03-16T17:57:42.736015+0000","last_clean":"2024-03-16T17:57:42.736015+0000","last_became_active":"2024-03-16T17:57:41.407916+0000","last_became_peered":"2024-03-16T17:57:41.407916+0000","last_unstale":"2024-03-16T17:57:42.736015+0000","last_undegraded":"2024-03-16T17:57:42.736015+0000","last_fullsized":"2024-03-16T17:57:42.736015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:36:37.396828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733788+0000","last_change":"2024-03-16T17:57:41.396339+0000","last_active":"2024-03-16T17:57:42.733788+0000","last_peered":"2024-03-16T17:57:42.733788+0000","last_clean":"2024-03-16T17:57:42.733788+0000","last_became_active":"2024-03-16T17:57:41.396068+0000","last_became_peered":"2024-03-16T17:57:41.396068+0000","last_unstale":"2024-03-16T17:57:42.733788+0000","last_undegraded":"2024-03-16T17:57:42.733788+0000","last_fullsized":"2024-03-16T17:57:42.733788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732607+0000","last_change":"2024-03-16T17:57:41.390555+0000","last_active":"2024-03-16T17:57:42.732607+0000","last_peered":"2024-03-16T17:57:42.732607+0000","last_clean":"2024-03-16T17:57:42.732607+0000","last_became_active":"2024-03-16T17:57:41.389886+0000","last_became_peered":"2024-03-16T17:57:41.389886+0000","last_unstale":"2024-03-16T17:57:42.732607+0000","last_undegraded":"2024-03-16T17:57:42.732607+0000","last_fullsized":"2024-03-16T17:57:42.732607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:20.628509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375544+0000","last_change":"2024-03-16T17:57:41.411237+0000","last_active":"2024-03-16T17:57:42.375544+0000","last_peered":"2024-03-16T17:57:42.375544+0000","last_clean":"2024-03-16T17:57:42.375544+0000","last_became_active":"2024-03-16T17:57:41.410757+0000","last_became_peered":"2024-03-16T17:57:41.410757+0000","last_unstale":"2024-03-16T17:57:42.375544+0000","last_undegraded":"2024-03-16T17:57:42.375544+0000","last_fullsized":"2024-03-16T17:57:42.375544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:01:51.467418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733707+0000","last_change":"2024-03-16T17:57:41.391770+0000","last_active":"2024-03-16T17:57:42.733707+0000","last_peered":"2024-03-16T17:57:42.733707+0000","last_clean":"2024-03-16T17:57:42.733707+0000","last_became_active":"2024-03-16T17:57:41.391470+0000","last_became_peered":"2024-03-16T17:57:41.391470+0000","last_unstale":"2024-03-16T17:57:42.733707+0000","last_undegraded":"2024-03-16T17:57:42.733707+0000","last_fullsized":"2024-03-16T17:57:42.733707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374329+0000","last_change":"2024-03-16T17:57:41.400595+0000","last_active":"2024-03-16T17:57:42.374329+0000","last_peered":"2024-03-16T17:57:42.374329+0000","last_clean":"2024-03-16T17:57:42.374329+0000","last_became_active":"2024-03-16T17:57:41.400295+0000","last_became_peered":"2024-03-16T17:57:41.400295+0000","last_unstale":"2024-03-16T17:57:42.374329+0000","last_undegraded":"2024-03-16T17:57:42.374329+0000","last_fullsized":"2024-03-16T17:57:42.374329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:55:42.207131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734977+0000","last_change":"2024-03-16T17:57:41.391182+0000","last_active":"2024-03-16T17:57:42.734977+0000","last_peered":"2024-03-16T17:57:42.734977+0000","last_clean":"2024-03-16T17:57:42.734977+0000","last_became_active":"2024-03-16T17:57:41.390811+0000","last_became_peered":"2024-03-16T17:57:41.390811+0000","last_unstale":"2024-03-16T17:57:42.734977+0000","last_undegraded":"2024-03-16T17:57:42.734977+0000","last_fullsized":"2024-03-16T17:57:42.734977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:55:42.207131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734433+0000","last_change":"2024-03-16T17:57:41.407541+0000","last_active":"2024-03-16T17:57:42.734433+0000","last_peered":"2024-03-16T17:57:42.734433+0000","last_clean":"2024-03-16T17:57:42.734433+0000","last_became_active":"2024-03-16T17:57:41.405546+0000","last_became_peered":"2024-03-16T17:57:41.405546+0000","last_unstale":"2024-03-16T17:57:42.734433+0000","last_undegraded":"2024-03-16T17:57:42.734433+0000","last_fullsized":"2024-03-16T17:57:42.734433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369831+0000","last_change":"2024-03-16T17:57:41.399134+0000","last_active":"2024-03-16T17:57:42.369831+0000","last_peered":"2024-03-16T17:57:42.369831+0000","last_clean":"2024-03-16T17:57:42.369831+0000","last_became_active":"2024-03-16T17:57:41.398841+0000","last_became_peered":"2024-03-16T17:57:41.398841+0000","last_unstale":"2024-03-16T17:57:42.369831+0000","last_undegraded":"2024-03-16T17:57:42.369831+0000","last_fullsized":"2024-03-16T17:57:42.369831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734641+0000","last_change":"2024-03-16T17:57:41.399114+0000","last_active":"2024-03-16T17:57:42.734641+0000","last_peered":"2024-03-16T17:57:42.734641+0000","last_clean":"2024-03-16T17:57:42.734641+0000","last_became_active":"2024-03-16T17:57:41.395668+0000","last_became_peered":"2024-03-16T17:57:41.395668+0000","last_unstale":"2024-03-16T17:57:42.734641+0000","last_undegraded":"2024-03-16T17:57:42.734641+0000","last_fullsized":"2024-03-16T17:57:42.734641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732560+0000","last_change":"2024-03-16T17:57:41.390678+0000","last_active":"2024-03-16T17:57:42.732560+0000","last_peered":"2024-03-16T17:57:42.732560+0000","last_clean":"2024-03-16T17:57:42.732560+0000","last_became_active":"2024-03-16T17:57:41.387968+0000","last_became_peered":"2024-03-16T17:57:41.387968+0000","last_unstale":"2024-03-16T17:57:42.732560+0000","last_undegraded":"2024-03-16T17:57:42.732560+0000","last_fullsized":"2024-03-16T17:57:42.732560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:59:12.277190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375711+0000","last_change":"2024-03-16T17:57:41.399083+0000","last_active":"2024-03-16T17:57:42.375711+0000","last_peered":"2024-03-16T17:57:42.375711+0000","last_clean":"2024-03-16T17:57:42.375711+0000","last_became_active":"2024-03-16T17:57:41.398899+0000","last_became_peered":"2024-03-16T17:57:41.398899+0000","last_unstale":"2024-03-16T17:57:42.375711+0000","last_undegraded":"2024-03-16T17:57:42.375711+0000","last_fullsized":"2024-03-16T17:57:42.375711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:10:39.722613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733367+0000","last_change":"2024-03-16T17:57:41.390700+0000","last_active":"2024-03-16T17:57:42.733367+0000","last_peered":"2024-03-16T17:57:42.733367+0000","last_clean":"2024-03-16T17:57:42.733367+0000","last_became_active":"2024-03-16T17:57:41.389732+0000","last_became_peered":"2024-03-16T17:57:41.389732+0000","last_unstale":"2024-03-16T17:57:42.733367+0000","last_undegraded":"2024-03-16T17:57:42.733367+0000","last_fullsized":"2024-03-16T17:57:42.733367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:32:52.837630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732990+0000","last_change":"2024-03-16T17:57:41.392698+0000","last_active":"2024-03-16T17:57:42.732990+0000","last_peered":"2024-03-16T17:57:42.732990+0000","last_clean":"2024-03-16T17:57:42.732990+0000","last_became_active":"2024-03-16T17:57:41.391950+0000","last_became_peered":"2024-03-16T17:57:41.391950+0000","last_unstale":"2024-03-16T17:57:42.732990+0000","last_undegraded":"2024-03-16T17:57:42.732990+0000","last_fullsized":"2024-03-16T17:57:42.732990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.372924+0000","last_change":"2024-03-16T17:57:41.383046+0000","last_active":"2024-03-16T17:57:42.372924+0000","last_peered":"2024-03-16T17:57:42.372924+0000","last_clean":"2024-03-16T17:57:42.372924+0000","last_became_active":"2024-03-16T17:57:41.382533+0000","last_became_peered":"2024-03-16T17:57:41.382533+0000","last_unstale":"2024-03-16T17:57:42.372924+0000","last_undegraded":"2024-03-16T17:57:42.372924+0000","last_fullsized":"2024-03-16T17:57:42.372924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:44:11.757790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734496+0000","last_change":"2024-03-16T17:57:40.368183+0000","last_active":"2024-03-16T17:57:42.734496+0000","last_peered":"2024-03-16T17:57:42.734496+0000","last_clean":"2024-03-16T17:57:42.734496+0000","last_became_active":"2024-03-16T17:57:40.367738+0000","last_became_peered":"2024-03-16T17:57:40.367738+0000","last_unstale":"2024-03-16T17:57:42.734496+0000","last_undegraded":"2024-03-16T17:57:42.734496+0000","last_fullsized":"2024-03-16T17:57:42.734496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732742+0000","last_change":"2024-03-16T17:57:41.378107+0000","last_active":"2024-03-16T17:57:42.732742+0000","last_peered":"2024-03-16T17:57:42.732742+0000","last_clean":"2024-03-16T17:57:42.732742+0000","last_became_active":"2024-03-16T17:57:41.375872+0000","last_became_peered":"2024-03-16T17:57:41.375872+0000","last_unstale":"2024-03-16T17:57:42.732742+0000","last_undegraded":"2024-03-16T17:57:42.732742+0000","last_fullsized":"2024-03-16T17:57:42.732742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733377+0000","last_change":"2024-03-16T17:57:40.375279+0000","last_active":"2024-03-16T17:57:42.733377+0000","last_peered":"2024-03-16T17:57:42.733377+0000","last_clean":"2024-03-16T17:57:42.733377+0000","last_became_active":"2024-03-16T17:57:40.374936+0000","last_became_peered":"2024-03-16T17:57:40.374936+0000","last_unstale":"2024-03-16T17:57:42.733377+0000","last_undegraded":"2024-03-16T17:57:42.733377+0000","last_fullsized":"2024-03-16T17:57:42.733377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:07.628213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733869+0000","last_change":"2024-03-16T17:57:40.372178+0000","last_active":"2024-03-16T17:57:42.733869+0000","last_peered":"2024-03-16T17:57:42.733869+0000","last_clean":"2024-03-16T17:57:42.733869+0000","last_became_active":"2024-03-16T17:57:40.370709+0000","last_became_peered":"2024-03-16T17:57:40.370709+0000","last_unstale":"2024-03-16T17:57:42.733869+0000","last_undegraded":"2024-03-16T17:57:42.733869+0000","last_fullsized":"2024-03-16T17:57:42.733869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:23:36.985797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732131+0000","last_change":"2024-03-16T17:57:41.397372+0000","last_active":"2024-03-16T17:57:42.732131+0000","last_peered":"2024-03-16T17:57:42.732131+0000","last_clean":"2024-03-16T17:57:42.732131+0000","last_became_active":"2024-03-16T17:57:41.397032+0000","last_became_peered":"2024-03-16T17:57:41.397032+0000","last_unstale":"2024-03-16T17:57:42.732131+0000","last_undegraded":"2024-03-16T17:57:42.732131+0000","last_fullsized":"2024-03-16T17:57:42.732131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374381+0000","last_change":"2024-03-16T17:57:40.362691+0000","last_active":"2024-03-16T17:57:42.374381+0000","last_peered":"2024-03-16T17:57:42.374381+0000","last_clean":"2024-03-16T17:57:42.374381+0000","last_became_active":"2024-03-16T17:57:40.361668+0000","last_became_peered":"2024-03-16T17:57:40.361668+0000","last_unstale":"2024-03-16T17:57:42.374381+0000","last_undegraded":"2024-03-16T17:57:42.374381+0000","last_fullsized":"2024-03-16T17:57:42.374381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369126+0000","last_change":"2024-03-16T17:57:41.393663+0000","last_active":"2024-03-16T17:57:42.369126+0000","last_peered":"2024-03-16T17:57:42.369126+0000","last_clean":"2024-03-16T17:57:42.369126+0000","last_became_active":"2024-03-16T17:57:41.393327+0000","last_became_peered":"2024-03-16T17:57:41.393327+0000","last_unstale":"2024-03-16T17:57:42.369126+0000","last_undegraded":"2024-03-16T17:57:42.369126+0000","last_fullsized":"2024-03-16T17:57:42.369126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733784+0000","last_change":"2024-03-16T17:57:40.377710+0000","last_active":"2024-03-16T17:57:42.733784+0000","last_peered":"2024-03-16T17:57:42.733784+0000","last_clean":"2024-03-16T17:57:42.733784+0000","last_became_active":"2024-03-16T17:57:40.377427+0000","last_became_peered":"2024-03-16T17:57:40.377427+0000","last_unstale":"2024-03-16T17:57:42.733784+0000","last_undegraded":"2024-03-16T17:57:42.733784+0000","last_fullsized":"2024-03-16T17:57:42.733784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376320+0000","last_change":"2024-03-16T17:57:41.413272+0000","last_active":"2024-03-16T17:57:42.376320+0000","last_peered":"2024-03-16T17:57:42.376320+0000","last_clean":"2024-03-16T17:57:42.376320+0000","last_became_active":"2024-03-16T17:57:41.412989+0000","last_became_peered":"2024-03-16T17:57:41.412989+0000","last_unstale":"2024-03-16T17:57:42.376320+0000","last_undegraded":"2024-03-16T17:57:42.376320+0000","last_fullsized":"2024-03-16T17:57:42.376320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:36:37.396828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376311+0000","last_change":"2024-03-16T17:57:40.380326+0000","last_active":"2024-03-16T17:57:42.376311+0000","last_peered":"2024-03-16T17:57:42.376311+0000","last_clean":"2024-03-16T17:57:42.376311+0000","last_became_active":"2024-03-16T17:57:40.380003+0000","last_became_peered":"2024-03-16T17:57:40.380003+0000","last_unstale":"2024-03-16T17:57:42.376311+0000","last_undegraded":"2024-03-16T17:57:42.376311+0000","last_fullsized":"2024-03-16T17:57:42.376311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:07.628213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733226+0000","last_change":"2024-03-16T17:57:41.407675+0000","last_active":"2024-03-16T17:57:42.733226+0000","last_peered":"2024-03-16T17:57:42.733226+0000","last_clean":"2024-03-16T17:57:42.733226+0000","last_became_active":"2024-03-16T17:57:41.407412+0000","last_became_peered":"2024-03-16T17:57:41.407412+0000","last_unstale":"2024-03-16T17:57:42.733226+0000","last_undegraded":"2024-03-16T17:57:42.733226+0000","last_fullsized":"2024-03-16T17:57:42.733226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370065+0000","last_change":"2024-03-16T17:57:40.375361+0000","last_active":"2024-03-16T17:57:42.370065+0000","last_peered":"2024-03-16T17:57:42.370065+0000","last_clean":"2024-03-16T17:57:42.370065+0000","last_became_active":"2024-03-16T17:57:40.375086+0000","last_became_peered":"2024-03-16T17:57:40.375086+0000","last_unstale":"2024-03-16T17:57:42.370065+0000","last_undegraded":"2024-03-16T17:57:42.370065+0000","last_fullsized":"2024-03-16T17:57:42.370065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373846+0000","last_change":"2024-03-16T17:57:41.404551+0000","last_active":"2024-03-16T17:57:42.373846+0000","last_peered":"2024-03-16T17:57:42.373846+0000","last_clean":"2024-03-16T17:57:42.373846+0000","last_became_active":"2024-03-16T17:57:41.404266+0000","last_became_peered":"2024-03-16T17:57:41.404266+0000","last_unstale":"2024-03-16T17:57:42.373846+0000","last_undegraded":"2024-03-16T17:57:42.373846+0000","last_fullsized":"2024-03-16T17:57:42.373846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:55:54.818895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733842+0000","last_change":"2024-03-16T17:57:40.376223+0000","last_active":"2024-03-16T17:57:42.733842+0000","last_peered":"2024-03-16T17:57:42.733842+0000","last_clean":"2024-03-16T17:57:42.733842+0000","last_became_active":"2024-03-16T17:57:40.375873+0000","last_became_peered":"2024-03-16T17:57:40.375873+0000","last_unstale":"2024-03-16T17:57:42.733842+0000","last_undegraded":"2024-03-16T17:57:42.733842+0000","last_fullsized":"2024-03-16T17:57:42.733842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735456+0000","last_change":"2024-03-16T17:57:41.403063+0000","last_active":"2024-03-16T17:57:42.735456+0000","last_peered":"2024-03-16T17:57:42.735456+0000","last_clean":"2024-03-16T17:57:42.735456+0000","last_became_active":"2024-03-16T17:57:41.402736+0000","last_became_peered":"2024-03-16T17:57:41.402736+0000","last_unstale":"2024-03-16T17:57:42.735456+0000","last_undegraded":"2024-03-16T17:57:42.735456+0000","last_fullsized":"2024-03-16T17:57:42.735456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:10:39.722613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732825+0000","last_change":"2024-03-16T17:57:40.363660+0000","last_active":"2024-03-16T17:57:42.732825+0000","last_peered":"2024-03-16T17:57:42.732825+0000","last_clean":"2024-03-16T17:57:42.732825+0000","last_became_active":"2024-03-16T17:57:40.363245+0000","last_became_peered":"2024-03-16T17:57:40.363245+0000","last_unstale":"2024-03-16T17:57:42.732825+0000","last_undegraded":"2024-03-16T17:57:42.732825+0000","last_fullsized":"2024-03-16T17:57:42.732825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:45:57.586553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732613+0000","last_change":"2024-03-16T17:57:41.398106+0000","last_active":"2024-03-16T17:57:42.732613+0000","last_peered":"2024-03-16T17:57:42.732613+0000","last_clean":"2024-03-16T17:57:42.732613+0000","last_became_active":"2024-03-16T17:57:41.396446+0000","last_became_peered":"2024-03-16T17:57:41.396446+0000","last_unstale":"2024-03-16T17:57:42.732613+0000","last_undegraded":"2024-03-16T17:57:42.732613+0000","last_fullsized":"2024-03-16T17:57:42.732613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:42:47.316038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375786+0000","last_change":"2024-03-16T17:57:40.377535+0000","last_active":"2024-03-16T17:57:42.375786+0000","last_peered":"2024-03-16T17:57:42.375786+0000","last_clean":"2024-03-16T17:57:42.375786+0000","last_became_active":"2024-03-16T17:57:40.377105+0000","last_became_peered":"2024-03-16T17:57:40.377105+0000","last_unstale":"2024-03-16T17:57:42.375786+0000","last_undegraded":"2024-03-16T17:57:42.375786+0000","last_fullsized":"2024-03-16T17:57:42.375786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735111+0000","last_change":"2024-03-16T17:57:41.407322+0000","last_active":"2024-03-16T17:57:42.735111+0000","last_peered":"2024-03-16T17:57:42.735111+0000","last_clean":"2024-03-16T17:57:42.735111+0000","last_became_active":"2024-03-16T17:57:41.403879+0000","last_became_peered":"2024-03-16T17:57:41.403879+0000","last_unstale":"2024-03-16T17:57:42.735111+0000","last_undegraded":"2024-03-16T17:57:42.735111+0000","last_fullsized":"2024-03-16T17:57:42.735111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:16:17.454713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732837+0000","last_change":"2024-03-16T17:57:40.368238+0000","last_active":"2024-03-16T17:57:42.732837+0000","last_peered":"2024-03-16T17:57:42.732837+0000","last_clean":"2024-03-16T17:57:42.732837+0000","last_became_active":"2024-03-16T17:57:40.367367+0000","last_became_peered":"2024-03-16T17:57:40.367367+0000","last_unstale":"2024-03-16T17:57:42.732837+0000","last_undegraded":"2024-03-16T17:57:42.732837+0000","last_fullsized":"2024-03-16T17:57:42.732837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:26.104046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732994+0000","last_change":"2024-03-16T17:57:41.389244+0000","last_active":"2024-03-16T17:57:42.732994+0000","last_peered":"2024-03-16T17:57:42.732994+0000","last_clean":"2024-03-16T17:57:42.732994+0000","last_became_active":"2024-03-16T17:57:41.388858+0000","last_became_peered":"2024-03-16T17:57:41.388858+0000","last_unstale":"2024-03-16T17:57:42.732994+0000","last_undegraded":"2024-03-16T17:57:42.732994+0000","last_fullsized":"2024-03-16T17:57:42.732994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733311+0000","last_change":"2024-03-16T17:57:40.365096+0000","last_active":"2024-03-16T17:57:42.733311+0000","last_peered":"2024-03-16T17:57:42.733311+0000","last_clean":"2024-03-16T17:57:42.733311+0000","last_became_active":"2024-03-16T17:57:40.364700+0000","last_became_peered":"2024-03-16T17:57:40.364700+0000","last_unstale":"2024-03-16T17:57:42.733311+0000","last_undegraded":"2024-03-16T17:57:42.733311+0000","last_fullsized":"2024-03-16T17:57:42.733311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:53:51.562074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734633+0000","last_change":"2024-03-16T17:57:41.405056+0000","last_active":"2024-03-16T17:57:42.734633+0000","last_peered":"2024-03-16T17:57:42.734633+0000","last_clean":"2024-03-16T17:57:42.734633+0000","last_became_active":"2024-03-16T17:57:41.403286+0000","last_became_peered":"2024-03-16T17:57:41.403286+0000","last_unstale":"2024-03-16T17:57:42.734633+0000","last_undegraded":"2024-03-16T17:57:42.734633+0000","last_fullsized":"2024-03-16T17:57:42.734633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374263+0000","last_change":"2024-03-16T17:57:40.365256+0000","last_active":"2024-03-16T17:57:42.374263+0000","last_peered":"2024-03-16T17:57:42.374263+0000","last_clean":"2024-03-16T17:57:42.374263+0000","last_became_active":"2024-03-16T17:57:40.362214+0000","last_became_peered":"2024-03-16T17:57:40.362214+0000","last_unstale":"2024-03-16T17:57:42.374263+0000","last_undegraded":"2024-03-16T17:57:42.374263+0000","last_fullsized":"2024-03-16T17:57:42.374263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:22.039976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735167+0000","last_change":"2024-03-16T17:57:41.413401+0000","last_active":"2024-03-16T17:57:42.735167+0000","last_peered":"2024-03-16T17:57:42.735167+0000","last_clean":"2024-03-16T17:57:42.735167+0000","last_became_active":"2024-03-16T17:57:41.413163+0000","last_became_peered":"2024-03-16T17:57:41.413163+0000","last_unstale":"2024-03-16T17:57:42.735167+0000","last_undegraded":"2024-03-16T17:57:42.735167+0000","last_fullsized":"2024-03-16T17:57:42.735167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:01:51.467418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734097+0000","last_change":"2024-03-16T17:57:40.370143+0000","last_active":"2024-03-16T17:57:42.734097+0000","last_peered":"2024-03-16T17:57:42.734097+0000","last_clean":"2024-03-16T17:57:42.734097+0000","last_became_active":"2024-03-16T17:57:40.369743+0000","last_became_peered":"2024-03-16T17:57:40.369743+0000","last_unstale":"2024-03-16T17:57:42.734097+0000","last_undegraded":"2024-03-16T17:57:42.734097+0000","last_fullsized":"2024-03-16T17:57:42.734097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:32:24.057779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736551+0000","last_change":"2024-03-16T17:57:41.413116+0000","last_active":"2024-03-16T17:57:42.736551+0000","last_peered":"2024-03-16T17:57:42.736551+0000","last_clean":"2024-03-16T17:57:42.736551+0000","last_became_active":"2024-03-16T17:57:41.412868+0000","last_became_peered":"2024-03-16T17:57:41.412868+0000","last_unstale":"2024-03-16T17:57:42.736551+0000","last_undegraded":"2024-03-16T17:57:42.736551+0000","last_fullsized":"2024-03-16T17:57:42.736551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:20:56.534791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370370+0000","last_change":"2024-03-16T17:57:40.364807+0000","last_active":"2024-03-16T17:57:42.370370+0000","last_peered":"2024-03-16T17:57:42.370370+0000","last_clean":"2024-03-16T17:57:42.370370+0000","last_became_active":"2024-03-16T17:57:40.363383+0000","last_became_peered":"2024-03-16T17:57:40.363383+0000","last_unstale":"2024-03-16T17:57:42.370370+0000","last_undegraded":"2024-03-16T17:57:42.370370+0000","last_fullsized":"2024-03-16T17:57:42.370370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:16.023603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375496+0000","last_change":"2024-03-16T17:57:41.402990+0000","last_active":"2024-03-16T17:57:42.375496+0000","last_peered":"2024-03-16T17:57:42.375496+0000","last_clean":"2024-03-16T17:57:42.375496+0000","last_became_active":"2024-03-16T17:57:41.402696+0000","last_became_peered":"2024-03-16T17:57:41.402696+0000","last_unstale":"2024-03-16T17:57:42.375496+0000","last_undegraded":"2024-03-16T17:57:42.375496+0000","last_fullsized":"2024-03-16T17:57:42.375496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:16:17.454713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733543+0000","last_change":"2024-03-16T17:57:40.378607+0000","last_active":"2024-03-16T17:57:42.733543+0000","last_peered":"2024-03-16T17:57:42.733543+0000","last_clean":"2024-03-16T17:57:42.733543+0000","last_became_active":"2024-03-16T17:57:40.378291+0000","last_became_peered":"2024-03-16T17:57:40.378291+0000","last_unstale":"2024-03-16T17:57:42.733543+0000","last_undegraded":"2024-03-16T17:57:42.733543+0000","last_fullsized":"2024-03-16T17:57:42.733543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:16.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735929+0000","last_change":"2024-03-16T17:57:41.409430+0000","last_active":"2024-03-16T17:57:42.735929+0000","last_peered":"2024-03-16T17:57:42.735929+0000","last_clean":"2024-03-16T17:57:42.735929+0000","last_became_active":"2024-03-16T17:57:41.409209+0000","last_became_peered":"2024-03-16T17:57:41.409209+0000","last_unstale":"2024-03-16T17:57:42.735929+0000","last_undegraded":"2024-03-16T17:57:42.735929+0000","last_fullsized":"2024-03-16T17:57:42.735929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:58.061341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733228+0000","last_change":"2024-03-16T17:57:35.560172+0000","last_active":"2024-03-16T17:57:42.733228+0000","last_peered":"2024-03-16T17:57:42.733228+0000","last_clean":"2024-03-16T17:57:42.733228+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T17:57:42.733228+0000","last_undegraded":"2024-03-16T17:57:42.733228+0000","last_fullsized":"2024-03-16T17:57:42.733228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_clean_scrub_stamp":"2024-03-16T17:57:34.174437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:20:03.039404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374626+0000","last_change":"2024-03-16T17:57:40.372426+0000","last_active":"2024-03-16T17:57:42.374626+0000","last_peered":"2024-03-16T17:57:42.374626+0000","last_clean":"2024-03-16T17:57:42.374626+0000","last_became_active":"2024-03-16T17:57:40.372097+0000","last_became_peered":"2024-03-16T17:57:40.372097+0000","last_unstale":"2024-03-16T17:57:42.374626+0000","last_undegraded":"2024-03-16T17:57:42.374626+0000","last_fullsized":"2024-03-16T17:57:42.374626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734031+0000","last_change":"2024-03-16T17:57:41.401095+0000","last_active":"2024-03-16T17:57:42.734031+0000","last_peered":"2024-03-16T17:57:42.734031+0000","last_clean":"2024-03-16T17:57:42.734031+0000","last_became_active":"2024-03-16T17:57:41.400788+0000","last_became_peered":"2024-03-16T17:57:41.400788+0000","last_unstale":"2024-03-16T17:57:42.734031+0000","last_undegraded":"2024-03-16T17:57:42.734031+0000","last_fullsized":"2024-03-16T17:57:42.734031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733259+0000","last_change":"2024-03-16T17:57:40.369748+0000","last_active":"2024-03-16T17:57:42.733259+0000","last_peered":"2024-03-16T17:57:42.733259+0000","last_clean":"2024-03-16T17:57:42.733259+0000","last_became_active":"2024-03-16T17:57:40.368850+0000","last_became_peered":"2024-03-16T17:57:40.368850+0000","last_unstale":"2024-03-16T17:57:42.733259+0000","last_undegraded":"2024-03-16T17:57:42.733259+0000","last_fullsized":"2024-03-16T17:57:42.733259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:31:16.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732283+0000","last_change":"2024-03-16T17:57:41.373872+0000","last_active":"2024-03-16T17:57:42.732283+0000","last_peered":"2024-03-16T17:57:42.732283+0000","last_clean":"2024-03-16T17:57:42.732283+0000","last_became_active":"2024-03-16T17:57:41.373478+0000","last_became_peered":"2024-03-16T17:57:41.373478+0000","last_unstale":"2024-03-16T17:57:42.732283+0000","last_undegraded":"2024-03-16T17:57:42.732283+0000","last_fullsized":"2024-03-16T17:57:42.732283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:04:23.189434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733503+0000","last_change":"2024-03-16T17:57:40.373316+0000","last_active":"2024-03-16T17:57:42.733503+0000","last_peered":"2024-03-16T17:57:42.733503+0000","last_clean":"2024-03-16T17:57:42.733503+0000","last_became_active":"2024-03-16T17:57:40.373015+0000","last_became_peered":"2024-03-16T17:57:40.373015+0000","last_unstale":"2024-03-16T17:57:42.733503+0000","last_undegraded":"2024-03-16T17:57:42.733503+0000","last_fullsized":"2024-03-16T17:57:42.733503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:42:38.164765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369425+0000","last_change":"2024-03-16T17:57:41.383198+0000","last_active":"2024-03-16T17:57:42.369425+0000","last_peered":"2024-03-16T17:57:42.369425+0000","last_clean":"2024-03-16T17:57:42.369425+0000","last_became_active":"2024-03-16T17:57:41.382886+0000","last_became_peered":"2024-03-16T17:57:41.382886+0000","last_unstale":"2024-03-16T17:57:42.369425+0000","last_undegraded":"2024-03-16T17:57:42.369425+0000","last_fullsized":"2024-03-16T17:57:42.369425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375951+0000","last_change":"2024-03-16T17:57:40.368934+0000","last_active":"2024-03-16T17:57:42.375951+0000","last_peered":"2024-03-16T17:57:42.375951+0000","last_clean":"2024-03-16T17:57:42.375951+0000","last_became_active":"2024-03-16T17:57:40.368496+0000","last_became_peered":"2024-03-16T17:57:40.368496+0000","last_unstale":"2024-03-16T17:57:42.375951+0000","last_undegraded":"2024-03-16T17:57:42.375951+0000","last_fullsized":"2024-03-16T17:57:42.375951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373158+0000","last_change":"2024-03-16T17:57:41.377235+0000","last_active":"2024-03-16T17:57:42.373158+0000","last_peered":"2024-03-16T17:57:42.373158+0000","last_clean":"2024-03-16T17:57:42.373158+0000","last_became_active":"2024-03-16T17:57:41.376796+0000","last_became_peered":"2024-03-16T17:57:41.376796+0000","last_unstale":"2024-03-16T17:57:42.373158+0000","last_undegraded":"2024-03-16T17:57:42.373158+0000","last_fullsized":"2024-03-16T17:57:42.373158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:24:23.606350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369894+0000","last_change":"2024-03-16T17:57:40.365747+0000","last_active":"2024-03-16T17:57:42.369894+0000","last_peered":"2024-03-16T17:57:42.369894+0000","last_clean":"2024-03-16T17:57:42.369894+0000","last_became_active":"2024-03-16T17:57:40.364579+0000","last_became_peered":"2024-03-16T17:57:40.364579+0000","last_unstale":"2024-03-16T17:57:42.369894+0000","last_undegraded":"2024-03-16T17:57:42.369894+0000","last_fullsized":"2024-03-16T17:57:42.369894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376270+0000","last_change":"2024-03-16T17:57:41.404470+0000","last_active":"2024-03-16T17:57:42.376270+0000","last_peered":"2024-03-16T17:57:42.376270+0000","last_clean":"2024-03-16T17:57:42.376270+0000","last_became_active":"2024-03-16T17:57:41.404100+0000","last_became_peered":"2024-03-16T17:57:41.404100+0000","last_unstale":"2024-03-16T17:57:42.376270+0000","last_undegraded":"2024-03-16T17:57:42.376270+0000","last_fullsized":"2024-03-16T17:57:42.376270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:50:09.742348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736227+0000","last_change":"2024-03-16T17:57:40.370118+0000","last_active":"2024-03-16T17:57:42.736227+0000","last_peered":"2024-03-16T17:57:42.736227+0000","last_clean":"2024-03-16T17:57:42.736227+0000","last_became_active":"2024-03-16T17:57:40.369751+0000","last_became_peered":"2024-03-16T17:57:40.369751+0000","last_unstale":"2024-03-16T17:57:42.736227+0000","last_undegraded":"2024-03-16T17:57:42.736227+0000","last_fullsized":"2024-03-16T17:57:42.736227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:04:22.171852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369793+0000","last_change":"2024-03-16T17:57:41.384659+0000","last_active":"2024-03-16T17:57:42.369793+0000","last_peered":"2024-03-16T17:57:42.369793+0000","last_clean":"2024-03-16T17:57:42.369793+0000","last_became_active":"2024-03-16T17:57:41.384374+0000","last_became_peered":"2024-03-16T17:57:41.384374+0000","last_unstale":"2024-03-16T17:57:42.369793+0000","last_undegraded":"2024-03-16T17:57:42.369793+0000","last_fullsized":"2024-03-16T17:57:42.369793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370123+0000","last_change":"2024-03-16T17:57:40.377982+0000","last_active":"2024-03-16T17:57:42.370123+0000","last_peered":"2024-03-16T17:57:42.370123+0000","last_clean":"2024-03-16T17:57:42.370123+0000","last_became_active":"2024-03-16T17:57:40.377669+0000","last_became_peered":"2024-03-16T17:57:40.377669+0000","last_unstale":"2024-03-16T17:57:42.370123+0000","last_undegraded":"2024-03-16T17:57:42.370123+0000","last_fullsized":"2024-03-16T17:57:42.370123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:31:16.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735021+0000","last_change":"2024-03-16T17:57:41.400625+0000","last_active":"2024-03-16T17:57:42.735021+0000","last_peered":"2024-03-16T17:57:42.735021+0000","last_clean":"2024-03-16T17:57:42.735021+0000","last_became_active":"2024-03-16T17:57:41.398361+0000","last_became_peered":"2024-03-16T17:57:41.398361+0000","last_unstale":"2024-03-16T17:57:42.735021+0000","last_undegraded":"2024-03-16T17:57:42.735021+0000","last_fullsized":"2024-03-16T17:57:42.735021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733837+0000","last_change":"2024-03-16T17:57:40.379426+0000","last_active":"2024-03-16T17:57:42.733837+0000","last_peered":"2024-03-16T17:57:42.733837+0000","last_clean":"2024-03-16T17:57:42.733837+0000","last_became_active":"2024-03-16T17:57:40.379136+0000","last_became_peered":"2024-03-16T17:57:40.379136+0000","last_unstale":"2024-03-16T17:57:42.733837+0000","last_undegraded":"2024-03-16T17:57:42.733837+0000","last_fullsized":"2024-03-16T17:57:42.733837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732957+0000","last_change":"2024-03-16T17:57:41.379286+0000","last_active":"2024-03-16T17:57:42.732957+0000","last_peered":"2024-03-16T17:57:42.732957+0000","last_clean":"2024-03-16T17:57:42.732957+0000","last_became_active":"2024-03-16T17:57:41.378998+0000","last_became_peered":"2024-03-16T17:57:41.378998+0000","last_unstale":"2024-03-16T17:57:42.732957+0000","last_undegraded":"2024-03-16T17:57:42.732957+0000","last_fullsized":"2024-03-16T17:57:42.732957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732976+0000","last_change":"2024-03-16T17:57:40.369141+0000","last_active":"2024-03-16T17:57:42.732976+0000","last_peered":"2024-03-16T17:57:42.732976+0000","last_clean":"2024-03-16T17:57:42.732976+0000","last_became_active":"2024-03-16T17:57:40.368542+0000","last_became_peered":"2024-03-16T17:57:40.368542+0000","last_unstale":"2024-03-16T17:57:42.732976+0000","last_undegraded":"2024-03-16T17:57:42.732976+0000","last_fullsized":"2024-03-16T17:57:42.732976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.372617+0000","last_change":"2024-03-16T17:57:41.375830+0000","last_active":"2024-03-16T17:57:42.372617+0000","last_peered":"2024-03-16T17:57:42.372617+0000","last_clean":"2024-03-16T17:57:42.372617+0000","last_became_active":"2024-03-16T17:57:41.375515+0000","last_became_peered":"2024-03-16T17:57:41.375515+0000","last_unstale":"2024-03-16T17:57:42.372617+0000","last_undegraded":"2024-03-16T17:57:42.372617+0000","last_fullsized":"2024-03-16T17:57:42.372617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:32:25.075362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369244+0000","last_change":"2024-03-16T17:57:40.367142+0000","last_active":"2024-03-16T17:57:42.369244+0000","last_peered":"2024-03-16T17:57:42.369244+0000","last_clean":"2024-03-16T17:57:42.369244+0000","last_became_active":"2024-03-16T17:57:40.366778+0000","last_became_peered":"2024-03-16T17:57:40.366778+0000","last_unstale":"2024-03-16T17:57:42.369244+0000","last_undegraded":"2024-03-16T17:57:42.369244+0000","last_fullsized":"2024-03-16T17:57:42.369244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732297+0000","last_change":"2024-03-16T17:57:41.388886+0000","last_active":"2024-03-16T17:57:42.732297+0000","last_peered":"2024-03-16T17:57:42.732297+0000","last_clean":"2024-03-16T17:57:42.732297+0000","last_became_active":"2024-03-16T17:57:41.388588+0000","last_became_peered":"2024-03-16T17:57:41.388588+0000","last_unstale":"2024-03-16T17:57:42.732297+0000","last_undegraded":"2024-03-16T17:57:42.732297+0000","last_fullsized":"2024-03-16T17:57:42.732297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369599+0000","last_change":"2024-03-16T17:57:40.371549+0000","last_active":"2024-03-16T17:57:42.369599+0000","last_peered":"2024-03-16T17:57:42.369599+0000","last_clean":"2024-03-16T17:57:42.369599+0000","last_became_active":"2024-03-16T17:57:40.371257+0000","last_became_peered":"2024-03-16T17:57:40.371257+0000","last_unstale":"2024-03-16T17:57:42.369599+0000","last_undegraded":"2024-03-16T17:57:42.369599+0000","last_fullsized":"2024-03-16T17:57:42.369599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734385+0000","last_change":"2024-03-16T17:57:41.392961+0000","last_active":"2024-03-16T17:57:42.734385+0000","last_peered":"2024-03-16T17:57:42.734385+0000","last_clean":"2024-03-16T17:57:42.734385+0000","last_became_active":"2024-03-16T17:57:41.392665+0000","last_became_peered":"2024-03-16T17:57:41.392665+0000","last_unstale":"2024-03-16T17:57:42.734385+0000","last_undegraded":"2024-03-16T17:57:42.734385+0000","last_fullsized":"2024-03-16T17:57:42.734385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735443+0000","last_change":"2024-03-16T17:57:40.376135+0000","last_active":"2024-03-16T17:57:42.735443+0000","last_peered":"2024-03-16T17:57:42.735443+0000","last_clean":"2024-03-16T17:57:42.735443+0000","last_became_active":"2024-03-16T17:57:40.375292+0000","last_became_peered":"2024-03-16T17:57:40.375292+0000","last_unstale":"2024-03-16T17:57:42.735443+0000","last_undegraded":"2024-03-16T17:57:42.735443+0000","last_fullsized":"2024-03-16T17:57:42.735443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:16.023603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733288+0000","last_change":"2024-03-16T17:57:41.394268+0000","last_active":"2024-03-16T17:57:42.733288+0000","last_peered":"2024-03-16T17:57:42.733288+0000","last_clean":"2024-03-16T17:57:42.733288+0000","last_became_active":"2024-03-16T17:57:41.393941+0000","last_became_peered":"2024-03-16T17:57:41.393941+0000","last_unstale":"2024-03-16T17:57:42.733288+0000","last_undegraded":"2024-03-16T17:57:42.733288+0000","last_fullsized":"2024-03-16T17:57:42.733288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376119+0000","last_change":"2024-03-16T17:57:41.402175+0000","last_active":"2024-03-16T17:57:42.376119+0000","last_peered":"2024-03-16T17:57:42.376119+0000","last_clean":"2024-03-16T17:57:42.376119+0000","last_became_active":"2024-03-16T17:57:41.401851+0000","last_became_peered":"2024-03-16T17:57:41.401851+0000","last_unstale":"2024-03-16T17:57:42.376119+0000","last_undegraded":"2024-03-16T17:57:42.376119+0000","last_fullsized":"2024-03-16T17:57:42.376119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:58.061341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734977+0000","last_change":"2024-03-16T17:57:40.377332+0000","last_active":"2024-03-16T17:57:42.734977+0000","last_peered":"2024-03-16T17:57:42.734977+0000","last_clean":"2024-03-16T17:57:42.734977+0000","last_became_active":"2024-03-16T17:57:40.377028+0000","last_became_peered":"2024-03-16T17:57:40.377028+0000","last_unstale":"2024-03-16T17:57:42.734977+0000","last_undegraded":"2024-03-16T17:57:42.734977+0000","last_fullsized":"2024-03-16T17:57:42.734977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375244+0000","last_change":"2024-03-16T17:57:41.402689+0000","last_active":"2024-03-16T17:57:42.375244+0000","last_peered":"2024-03-16T17:57:42.375244+0000","last_clean":"2024-03-16T17:57:42.375244+0000","last_became_active":"2024-03-16T17:57:41.401253+0000","last_became_peered":"2024-03-16T17:57:41.401253+0000","last_unstale":"2024-03-16T17:57:42.375244+0000","last_undegraded":"2024-03-16T17:57:42.375244+0000","last_fullsized":"2024-03-16T17:57:42.375244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735773+0000","last_change":"2024-03-16T17:57:40.379396+0000","last_active":"2024-03-16T17:57:42.735773+0000","last_peered":"2024-03-16T17:57:42.735773+0000","last_clean":"2024-03-16T17:57:42.735773+0000","last_became_active":"2024-03-16T17:57:40.379058+0000","last_became_peered":"2024-03-16T17:57:40.379058+0000","last_unstale":"2024-03-16T17:57:42.735773+0000","last_undegraded":"2024-03-16T17:57:42.735773+0000","last_fullsized":"2024-03-16T17:57:42.735773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:53:59.451003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732999+0000","last_change":"2024-03-16T17:57:41.407792+0000","last_active":"2024-03-16T17:57:42.732999+0000","last_peered":"2024-03-16T17:57:42.732999+0000","last_clean":"2024-03-16T17:57:42.732999+0000","last_became_active":"2024-03-16T17:57:41.407412+0000","last_became_peered":"2024-03-16T17:57:41.407412+0000","last_unstale":"2024-03-16T17:57:42.732999+0000","last_undegraded":"2024-03-16T17:57:42.732999+0000","last_fullsized":"2024-03-16T17:57:42.732999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735117+0000","last_change":"2024-03-16T17:57:40.366569+0000","last_active":"2024-03-16T17:57:42.735117+0000","last_peered":"2024-03-16T17:57:42.735117+0000","last_clean":"2024-03-16T17:57:42.735117+0000","last_became_active":"2024-03-16T17:57:40.362328+0000","last_became_peered":"2024-03-16T17:57:40.362328+0000","last_unstale":"2024-03-16T17:57:42.735117+0000","last_undegraded":"2024-03-16T17:57:42.735117+0000","last_fullsized":"2024-03-16T17:57:42.735117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734985+0000","last_change":"2024-03-16T17:57:41.400212+0000","last_active":"2024-03-16T17:57:42.734985+0000","last_peered":"2024-03-16T17:57:42.734985+0000","last_clean":"2024-03-16T17:57:42.734985+0000","last_became_active":"2024-03-16T17:57:41.399937+0000","last_became_peered":"2024-03-16T17:57:41.399937+0000","last_unstale":"2024-03-16T17:57:42.734985+0000","last_undegraded":"2024-03-16T17:57:42.734985+0000","last_fullsized":"2024-03-16T17:57:42.734985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733309+0000","last_change":"2024-03-16T17:57:40.368635+0000","last_active":"2024-03-16T17:57:42.733309+0000","last_peered":"2024-03-16T17:57:42.733309+0000","last_clean":"2024-03-16T17:57:42.733309+0000","last_became_active":"2024-03-16T17:57:40.368370+0000","last_became_peered":"2024-03-16T17:57:40.368370+0000","last_unstale":"2024-03-16T17:57:42.733309+0000","last_undegraded":"2024-03-16T17:57:42.733309+0000","last_fullsized":"2024-03-16T17:57:42.733309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:19.918231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":798720,"data_stored":658813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":69475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":67953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542147,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":69475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542147,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T17:57:46.239 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-16T17:57:46.239 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-16T17:57:46.239 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-16T17:57:46.240 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-16T17:57:46.240 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-16T17:57:46.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.488+0000 7fa6a3400640 1 Processor -- start 2024-03-16T17:57:46.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.488+0000 7fa6a3400640 1 -- start start 2024-03-16T17:57:46.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.490+0000 7fa6a3400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa69c15e2c0 0x7fa69c0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.490+0000 7fa6a3400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69c15e790 con 0x7fa69c072a20 2024-03-16T17:57:46.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.490+0000 7fa6a3400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa69c15ea70 con 0x7fa69c15dee0 2024-03-16T17:57:46.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.490+0000 7fa6a3400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69c15ed50 con 0x7fa69c15e2c0 2024-03-16T17:57:46.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.490+0000 7fa69be00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa69c072a20 0x7fa69c0b3340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56160/0 (socket says 172.21.15.82:56160) 2024-03-16T17:57:46.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.490+0000 7fa69be00640 1 -- 172.21.15.82:0/2631415211 learned_addr learned my addr 172.21.15.82:0/2631415211 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:46.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa6a1400640 1 --2- 172.21.15.82:0/2631415211 >> v2:172.21.15.90:3300/0 conn(0x7fa69c15e2c0 0x7fa69c0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549965020 0 0) 0x7fa69c15e790 con 0x7fa69c072a20 2024-03-16T17:57:46.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa680003520 con 0x7fa69c072a20 2024-03-16T17:57:46.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 344584834 0 0) 0x7fa680003520 con 0x7fa69c072a20 2024-03-16T17:57:46.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 >> v2:172.21.15.90:3300/0 conn(0x7fa69c15e2c0 msgr2=0x7fa69c0b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:46.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 --2- 172.21.15.82:0/2631415211 >> v2:172.21.15.90:3300/0 conn(0x7fa69c15e2c0 0x7fa69c0b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 >> v1:172.21.15.90:6789/0 conn(0x7fa69c15dee0 legacy=0x7fa69c072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:46.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c15f030 con 0x7fa69c072a20 2024-03-16T17:57:46.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fa688001670 con 0x7fa69c072a20 2024-03-16T17:57:46.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 >> v1:172.21.15.82:6789/0 conn(0x7fa69c072a20 legacy=0x7fa69c0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa680003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6800047c0 0x7fa680004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa680005180 0x7fa68000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c15ed50 con 0x7fa680003970 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68000c020 con 0x7fa6800047c0 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.491+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68000c1e0 con 0x7fa680005180 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.492+0000 7fa6a0a00640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa680003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.492+0000 7fa69be00640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6800047c0 0x7fa680004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.492+0000 7fa6a0a00640 1 -- 172.21.15.82:0/2631415211 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa680005180 msgr2=0x7fa68000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.492+0000 7fa6a0a00640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa680005180 0x7fa68000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.492+0000 7fa6a0a00640 1 -- 172.21.15.82:0/2631415211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6800047c0 msgr2=0x7fa680004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.492+0000 7fa6a0a00640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6800047c0 0x7fa680004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.492+0000 7fa6a0a00640 1 -- 172.21.15.82:0/2631415211 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa680003520 con 0x7fa680003970 2024-03-16T17:57:46.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.492+0000 7fa6a1400640 1 -- 172.21.15.82:0/2631415211 reap_dead start 2024-03-16T17:57:46.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.492+0000 7fa69be00640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa6800047c0 0x7fa680004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:46.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.493+0000 7fa6a0a00640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa680003d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa68c001750 tx=0x7fa6800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3e7906a201036e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:46.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.493+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0052d0 con 0x7fa680003970 2024-03-16T17:57:46.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.493+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c02e050 con 0x7fa680003970 2024-03-16T17:57:46.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.493+0000 7fa69b400640 1 -- 172.21.15.82:0/2631415211 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c003830 con 0x7fa680003970 2024-03-16T17:57:46.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.494+0000 7fa6a3400640 1 -- 172.21.15.82:0/2631415211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 msgr2=0x7fa680003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:46.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.494+0000 7fa6a3400640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa680003d90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa68c001750 tx=0x7fa6800057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.494+0000 7fa6a3400640 1 -- 172.21.15.82:0/2631415211 shutdown_connections 2024-03-16T17:57:46.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.494+0000 7fa6a3400640 1 --2- 172.21.15.82:0/2631415211 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa680003d90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.494+0000 7fa6a3400640 1 -- 172.21.15.82:0/2631415211 >> 172.21.15.82:0/2631415211 conn(0x7fa69c06ae70 msgr2=0x7fa69c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:46.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.494+0000 7fa6a3400640 1 -- 172.21.15.82:0/2631415211 shutdown_connections 2024-03-16T17:57:46.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.494+0000 7fa6a3400640 1 -- 172.21.15.82:0/2631415211 wait complete. 2024-03-16T17:57:46.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.496+0000 7fa6a3400640 1 Processor -- start 2024-03-16T17:57:46.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.496+0000 7fa6a3400640 1 -- start start 2024-03-16T17:57:46.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa69c1f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a3400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa69c153cb0 0x7fa69c154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa69c1f2820 0x7fa69c1f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a3400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c0b2080 con 0x7fa680003970 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a3400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c0afd30 con 0x7fa69c1f2820 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a3400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c0b2290 con 0x7fa69c153cb0 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa69c1f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa69c1f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35132/0 (socket says 172.21.15.82:35132) 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a0a00640 1 -- 172.21.15.82:0/1772437875 learned_addr learned my addr 172.21.15.82:0/1772437875 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a1400640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa69c1f2820 0x7fa69c1f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa69be00640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa69c153cb0 0x7fa69c154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a0a00640 1 -- 172.21.15.82:0/1772437875 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa69c153cb0 msgr2=0x7fa69c154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a0a00640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa69c153cb0 0x7fa69c154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a0a00640 1 -- 172.21.15.82:0/1772437875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa69c1f2820 msgr2=0x7fa69c1f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:46.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a0a00640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa69c1f2820 0x7fa69c1f2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a0a00640 1 -- 172.21.15.82:0/1772437875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c0af770 con 0x7fa680003970 2024-03-16T17:57:46.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.497+0000 7fa6a1400640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa69c1f2820 0x7fa69c1f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:46.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.498+0000 7fa6a0a00640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa69c1f4480 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa68c008fd0 tx=0x7fa68c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:46.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.498+0000 7fa698c00640 1 -- 172.21.15.82:0/1772437875 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c0076c0 con 0x7fa680003970 2024-03-16T17:57:46.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.498+0000 7fa698c00640 1 -- 172.21.15.82:0/1772437875 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c007a30 con 0x7fa680003970 2024-03-16T17:57:46.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.498+0000 7fa6a3400640 1 -- 172.21.15.82:0/1772437875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69c0afa50 con 0x7fa680003970 2024-03-16T17:57:46.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.498+0000 7fa698c00640 1 -- 172.21.15.82:0/1772437875 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c006c00 con 0x7fa680003970 2024-03-16T17:57:46.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.498+0000 7fa6a3400640 1 -- 172.21.15.82:0/1772437875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69c1f31e0 con 0x7fa680003970 2024-03-16T17:57:46.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.499+0000 7fa6a3400640 1 -- 172.21.15.82:0/1772437875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69c0afec0 con 0x7fa680003970 2024-03-16T17:57:46.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.500+0000 7fa698c00640 1 -- 172.21.15.82:0/1772437875 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa68c0035f0 con 0x7fa680003970 2024-03-16T17:57:46.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.501+0000 7fa698c00640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa67c06b3b0 0x7fa67c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.502+0000 7fa698c00640 1 -- 172.21.15.82:0/1772437875 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa68c0ab9a0 con 0x7fa680003970 2024-03-16T17:57:46.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.502+0000 7fa69be00640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa67c06b3b0 0x7fa67c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.502+0000 7fa69be00640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa67c06b3b0 0x7fa67c06d870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa688000ff0 tx=0x7fa688009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:46.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.506+0000 7fa698c00640 1 -- 172.21.15.82:0/1772437875 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa69c0afec0 con 0x7fa680003970 2024-03-16T17:57:46.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.752+0000 7fa6a3400640 1 -- 172.21.15.82:0/1772437875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa69c0afe70 con 0x7fa680003970 2024-03-16T17:57:46.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.760+0000 7fa698c00640 1 -- 172.21.15.82:0/1772437875 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171244 (secure 0 0 0) 0x7fa68c04b090 con 0x7fa680003970 2024-03-16T17:57:46.767 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.764+0000 7fa6a3400640 1 -- 172.21.15.82:0/1772437875 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa67c06b3b0 msgr2=0x7fa67c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.764+0000 7fa6a3400640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa67c06b3b0 0x7fa67c06d870 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa688000ff0 tx=0x7fa688009040 comp rx=0 tx=0).stop 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.764+0000 7fa6a3400640 1 -- 172.21.15.82:0/1772437875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 msgr2=0x7fa69c1f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.764+0000 7fa6a3400640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa69c1f4480 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa68c008fd0 tx=0x7fa68c001120 comp rx=0 tx=0).stop 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.764+0000 7fa6a3400640 1 -- 172.21.15.82:0/1772437875 shutdown_connections 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.764+0000 7fa6a3400640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa67c06b3b0 0x7fa67c06d870 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.764+0000 7fa6a3400640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa69c1f2820 0x7fa69c1f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.764+0000 7fa6a3400640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa69c153cb0 0x7fa69c154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.764+0000 7fa6a3400640 1 --2- 172.21.15.82:0/1772437875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa680003970 0x7fa69c1f4480 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.764+0000 7fa6a3400640 1 -- 172.21.15.82:0/1772437875 >> 172.21.15.82:0/1772437875 conn(0x7fa69c06ae70 msgr2=0x7fa69c0712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:46.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.765+0000 7fa6a3400640 1 -- 172.21.15.82:0/1772437875 shutdown_connections 2024-03-16T17:57:46.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.765+0000 7fa6a3400640 1 -- 172.21.15.82:0/1772437875 wait complete. 2024-03-16T17:57:46.787 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":3774193927},{"type":"v1","addr":"172.21.15.82:6833","nonce":3774193927}]},"active_addr":"172.21.15.82:6833/3774193927","active_change":"2024-03-16T17:57:25.985947+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":1162541984}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":34257154}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":541167966}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":1141505646}]}]} 2024-03-16T17:57:46.791 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-16T17:57:46.791 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-16T17:57:46.791 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:46.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.989+0000 7f2f89a00640 1 Processor -- start 2024-03-16T17:57:46.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.989+0000 7f2f89a00640 1 -- start start 2024-03-16T17:57:46.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f89a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f841595b0 0x7f2f840ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f89a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2f84159990 0x7f2f840aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f89a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2f8406d030 0x7f2f8406d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f89a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f840b3370 con 0x7f2f8406d030 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f89a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f8406dd20 con 0x7f2f84159990 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f89a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8415f7a0 con 0x7f2f841595b0 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f89000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2f8406d030 0x7f2f8406d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f89000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2f8406d030 0x7f2f8406d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35136/0 (socket says 172.21.15.82:35136) 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f89000640 1 -- 172.21.15.82:0/3007054401 learned_addr learned my addr 172.21.15.82:0/3007054401 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f83e00640 1 --2- 172.21.15.82:0/3007054401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f841595b0 0x7f2f840ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.991+0000 7f2f7b400640 1 --2- 172.21.15.82:0/3007054401 >> v2:172.21.15.90:3300/0 conn(0x7f2f84159990 0x7f2f840aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.992+0000 7f2f89000640 1 -- 172.21.15.82:0/3007054401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f841595b0 msgr2=0x7f2f840ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.992+0000 7f2f89000640 1 --2- 172.21.15.82:0/3007054401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f841595b0 0x7f2f840ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.992+0000 7f2f89000640 1 -- 172.21.15.82:0/3007054401 >> v2:172.21.15.90:3300/0 conn(0x7f2f84159990 msgr2=0x7f2f840aec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.992+0000 7f2f89000640 1 --2- 172.21.15.82:0/3007054401 >> v2:172.21.15.90:3300/0 conn(0x7f2f84159990 0x7f2f840aec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.992+0000 7f2f89000640 1 -- 172.21.15.82:0/3007054401 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8415fa80 con 0x7f2f8406d030 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.993+0000 7f2f89000640 1 --2- 172.21.15.82:0/3007054401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f8406d030 0x7f2f8406d4c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f84072b40 tx=0x7f2f7c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41956601e8638acd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.993+0000 7f2f83400640 1 -- 172.21.15.82:0/3007054401 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c031860 con 0x7f2f8406d030 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.993+0000 7f2f83400640 1 -- 172.21.15.82:0/3007054401 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c0319c0 con 0x7f2f8406d030 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.993+0000 7f2f83400640 1 -- 172.21.15.82:0/3007054401 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c031d50 con 0x7f2f8406d030 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.994+0000 7f2f89a00640 1 -- 172.21.15.82:0/3007054401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f8406d030 msgr2=0x7f2f8406d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.994+0000 7f2f89a00640 1 --2- 172.21.15.82:0/3007054401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f8406d030 0x7f2f8406d4c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f84072b40 tx=0x7f2f7c02b900 comp rx=0 tx=0).stop 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.994+0000 7f2f89a00640 1 -- 172.21.15.82:0/3007054401 shutdown_connections 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.994+0000 7f2f89a00640 1 --2- 172.21.15.82:0/3007054401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f8406d030 0x7f2f8406d4c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.994+0000 7f2f89a00640 1 --2- 172.21.15.82:0/3007054401 >> v2:172.21.15.90:3300/0 conn(0x7f2f84159990 0x7f2f840aec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.994+0000 7f2f89a00640 1 --2- 172.21.15.82:0/3007054401 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f841595b0 0x7f2f840ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.994+0000 7f2f89a00640 1 -- 172.21.15.82:0/3007054401 >> 172.21.15.82:0/3007054401 conn(0x7f2f8406ae70 msgr2=0x7f2f8406b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:46.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.994+0000 7f2f89a00640 1 -- 172.21.15.82:0/3007054401 shutdown_connections 2024-03-16T17:57:46.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.994+0000 7f2f89a00640 1 -- 172.21.15.82:0/3007054401 wait complete. 2024-03-16T17:57:46.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.996+0000 7f2f89a00640 1 Processor -- start 2024-03-16T17:57:46.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.996+0000 7f2f89a00640 1 -- start start 2024-03-16T17:57:47.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.997+0000 7f2f89a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f8406d030 0x7f2f841482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.997+0000 7f2f89a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f841595b0 0x7f2f84072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.997+0000 7f2f89a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f84159990 0x7f2f8415d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.997+0000 7f2f89a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84071e70 con 0x7f2f841595b0 2024-03-16T17:57:47.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.997+0000 7f2f89a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84070fa0 con 0x7f2f84159990 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.997+0000 7f2f89a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f840711f0 con 0x7f2f8406d030 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.997+0000 7f2f7b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f841595b0 0x7f2f84072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.997+0000 7f2f7b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f841595b0 0x7f2f84072290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35138/0 (socket says 172.21.15.82:35138) 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.997+0000 7f2f7b400640 1 -- 172.21.15.82:0/2130230154 learned_addr learned my addr 172.21.15.82:0/2130230154 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.997+0000 7f2f83e00640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f8406d030 0x7f2f841482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f89000640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f84159990 0x7f2f8415d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f7b400640 1 -- 172.21.15.82:0/2130230154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f8406d030 msgr2=0x7f2f841482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f7b400640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f8406d030 0x7f2f841482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f7b400640 1 -- 172.21.15.82:0/2130230154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f84159990 msgr2=0x7f2f8415d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f7b400640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f84159990 0x7f2f8415d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f7b400640 1 -- 172.21.15.82:0/2130230154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c0094d0 con 0x7f2f841595b0 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f89000640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f84159990 0x7f2f8415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:47.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f7b400640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f841595b0 0x7f2f84072290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00d5f0 tx=0x7f2f6c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:47.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f80c00640 1 -- 172.21.15.82:0/2130230154 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0125f0 con 0x7f2f841595b0 2024-03-16T17:57:47.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f89a00640 1 -- 172.21.15.82:0/2130230154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f840714d0 con 0x7f2f841595b0 2024-03-16T17:57:47.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f80c00640 1 -- 172.21.15.82:0/2130230154 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c012750 con 0x7f2f841595b0 2024-03-16T17:57:47.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f80c00640 1 -- 172.21.15.82:0/2130230154 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c012ac0 con 0x7f2f841595b0 2024-03-16T17:57:47.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:46.998+0000 7f2f89a00640 1 -- 172.21.15.82:0/2130230154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f84071a90 con 0x7f2f841595b0 2024-03-16T17:57:47.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.000+0000 7f2f89a00640 1 -- 172.21.15.82:0/2130230154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f84071fb0 con 0x7f2f841595b0 2024-03-16T17:57:47.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.000+0000 7f2f80c00640 1 -- 172.21.15.82:0/2130230154 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f2f6c012c20 con 0x7f2f841595b0 2024-03-16T17:57:47.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.001+0000 7f2f80c00640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f6006b3b0 0x7f2f6006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.001+0000 7f2f80c00640 1 -- 172.21.15.82:0/2130230154 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f6c089f70 con 0x7f2f841595b0 2024-03-16T17:57:47.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.002+0000 7f2f83e00640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f6006b3b0 0x7f2f6006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.002+0000 7f2f83e00640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f6006b3b0 0x7f2f6006d870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2f840ae590 tx=0x7f2f740039b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:47.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.007+0000 7f2f80c00640 1 -- 172.21.15.82:0/2130230154 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2f84071fb0 con 0x7f2f841595b0 2024-03-16T17:57:47.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.207+0000 7f2f89a00640 1 -- 172.21.15.82:0/2130230154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f84071fb0 con 0x7f2f841595b0 2024-03-16T17:57:47.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.210+0000 7f2f80c00640 1 -- 172.21.15.82:0/2130230154 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f2f84071fb0 con 0x7f2f841595b0 2024-03-16T17:57:47.213 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:47.214 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:47.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.212+0000 7f2f89a00640 1 -- 172.21.15.82:0/2130230154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f6006b3b0 msgr2=0x7f2f6006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.212+0000 7f2f89a00640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f6006b3b0 0x7f2f6006d870 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2f840ae590 tx=0x7f2f740039b0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.212+0000 7f2f89a00640 1 -- 172.21.15.82:0/2130230154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f841595b0 msgr2=0x7f2f84072290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.212+0000 7f2f89a00640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f841595b0 0x7f2f84072290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00d5f0 tx=0x7f2f6c00dab0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.212+0000 7f2f89a00640 1 -- 172.21.15.82:0/2130230154 shutdown_connections 2024-03-16T17:57:47.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.212+0000 7f2f89a00640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f6006b3b0 0x7f2f6006d870 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.212+0000 7f2f89a00640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f84159990 0x7f2f8415d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.212+0000 7f2f89a00640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f841595b0 0x7f2f84072290 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.212+0000 7f2f89a00640 1 --2- 172.21.15.82:0/2130230154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f8406d030 0x7f2f841482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.212+0000 7f2f89a00640 1 -- 172.21.15.82:0/2130230154 >> 172.21.15.82:0/2130230154 conn(0x7f2f8406ae70 msgr2=0x7f2f84159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:47.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.213+0000 7f2f89a00640 1 -- 172.21.15.82:0/2130230154 shutdown_connections 2024-03-16T17:57:47.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.213+0000 7f2f89a00640 1 -- 172.21.15.82:0/2130230154 wait complete. 2024-03-16T17:57:47.235 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-16T17:57:47.236 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:47.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.441+0000 7fcff3000640 1 Processor -- start 2024-03-16T17:57:47.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.441+0000 7fcff3000640 1 -- start start 2024-03-16T17:57:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.443+0000 7fcff3000640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fcfec144400 0x7fcfec0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.443+0000 7fcff3000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec15ba20 0x7fcfec0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.443+0000 7fcff3000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec15e820 con 0x7fcfec15f550 2024-03-16T17:57:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.443+0000 7fcff3000640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfec15f130 con 0x7fcfec144400 2024-03-16T17:57:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.443+0000 7fcff3000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec15be60 con 0x7fcfec15ba20 2024-03-16T17:57:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeb400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcfec15f550 0x7fcfec15b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56172/0 (socket says 172.21.15.82:56172) 2024-03-16T17:57:47.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeb400640 1 -- 172.21.15.82:0/3595549628 learned_addr learned my addr 172.21.15.82:0/3595549628 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcff1000640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec15ba20 0x7fcfec0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfebe00640 1 --2- 172.21.15.82:0/3595549628 >> v2:172.21.15.90:3300/0 conn(0x7fcfec144400 0x7fcfec0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460734115 0 0) 0x7fcfec15e820 con 0x7fcfec15f550 2024-03-16T17:57:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfd0005730 con 0x7fcfec15f550 2024-03-16T17:57:47.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3650005197 0 0) 0x7fcfd0005730 con 0x7fcfec15f550 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec15ba20 msgr2=0x7fcfec0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec15ba20 0x7fcfec0cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 >> v2:172.21.15.90:3300/0 conn(0x7fcfec144400 msgr2=0x7fcfec0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 --2- 172.21.15.82:0/3595549628 >> v2:172.21.15.90:3300/0 conn(0x7fcfec144400 0x7fcfec0cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec15c080 con 0x7fcfec15f550 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fcfd8001670 con 0x7fcfec15f550 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 >> v1:172.21.15.82:6789/0 conn(0x7fcfec15f550 legacy=0x7fcfec15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.444+0000 7fcfeaa00640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec15ba20 0x7fcfd0005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeaa00640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 0x7fcfd00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeaa00640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd0006f10 0x7fcfd0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec15f130 con 0x7fcfd0006130 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec15be60 con 0x7fcfd0006f10 2024-03-16T17:57:47.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd000dc10 con 0x7fcfec15ba20 2024-03-16T17:57:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeb400640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 0x7fcfd00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfebe00640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec15ba20 0x7fcfd0005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcff1000640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd0006f10 0x7fcfd0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeb400640 1 -- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec15ba20 msgr2=0x7fcfd0005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeb400640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec15ba20 0x7fcfd0005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeb400640 1 -- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd0006f10 msgr2=0x7fcfd0007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeb400640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd0006f10 0x7fcfd0007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.445+0000 7fcfeb400640 1 -- 172.21.15.82:0/3595549628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd0005730 con 0x7fcfd0006130 2024-03-16T17:57:47.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.446+0000 7fcfeb400640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 0x7fcfd00069d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800c240 tx=0x7fcfd0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3adda1f96df43e4d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.446+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8005ad0 con 0x7fcfd0006130 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.447+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd8005c30 con 0x7fcfd0006130 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.447+0000 7fcfeaa00640 1 -- 172.21.15.82:0/3595549628 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd802c8b0 con 0x7fcfd0006130 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.447+0000 7fcff3000640 1 -- 172.21.15.82:0/3595549628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 msgr2=0x7fcfd00069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.447+0000 7fcff3000640 1 --2- 172.21.15.82:0/3595549628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 0x7fcfd00069d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800c240 tx=0x7fcfd0007a00 comp rx=0 tx=0).stop 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.447+0000 7fcff1000640 1 -- 172.21.15.82:0/3595549628 reap_dead start 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.447+0000 7fcff3000640 1 -- 172.21.15.82:0/3595549628 shutdown_connections 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.447+0000 7fcff3000640 1 -- 172.21.15.82:0/3595549628 >> 172.21.15.82:0/3595549628 conn(0x7fcfec06ae70 msgr2=0x7fcfec06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.447+0000 7fcff3000640 1 -- 172.21.15.82:0/3595549628 shutdown_connections 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.447+0000 7fcff3000640 1 -- 172.21.15.82:0/3595549628 wait complete. 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.449+0000 7fcff3000640 1 Processor -- start 2024-03-16T17:57:47.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.449+0000 7fcff3000640 1 -- start start 2024-03-16T17:57:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.450+0000 7fcff3000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 0x7fcfec0aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.450+0000 7fcff3000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfec0b01e0 0x7fcfec1f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.450+0000 7fcff3000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec1f7780 0x7fcfec0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.450+0000 7fcff3000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec1f4820 con 0x7fcfd0006130 2024-03-16T17:57:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.450+0000 7fcff3000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec1f72f0 con 0x7fcfec0b01e0 2024-03-16T17:57:47.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.450+0000 7fcff3000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0b0660 con 0x7fcfec1f7780 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.450+0000 7fcfebe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 0x7fcfec0aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.450+0000 7fcfebe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 0x7fcfec0aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35156/0 (socket says 172.21.15.82:35156) 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.450+0000 7fcfebe00640 1 -- 172.21.15.82:0/189685706 learned_addr learned my addr 172.21.15.82:0/189685706 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcff1000640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec1f7780 0x7fcfec0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfebe00640 1 -- 172.21.15.82:0/189685706 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec1f7780 msgr2=0x7fcfec0710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfeb400640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfec0b01e0 0x7fcfec1f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfebe00640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec1f7780 0x7fcfec0710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfebe00640 1 -- 172.21.15.82:0/189685706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfec0b01e0 msgr2=0x7fcfec1f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfebe00640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfec0b01e0 0x7fcfec1f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfebe00640 1 -- 172.21.15.82:0/189685706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0b08e0 con 0x7fcfd0006130 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfeb400640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfec0b01e0 0x7fcfec1f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfebe00640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 0x7fcfec0aee90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc011630 tx=0x7fcfdc011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfe3e00640 1 -- 172.21.15.82:0/189685706 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc00f920 con 0x7fcfd0006130 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfe3e00640 1 -- 172.21.15.82:0/189685706 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc00fa80 con 0x7fcfd0006130 2024-03-16T17:57:47.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcfe3e00640 1 -- 172.21.15.82:0/189685706 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc00fdf0 con 0x7fcfd0006130 2024-03-16T17:57:47.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.451+0000 7fcff3000640 1 -- 172.21.15.82:0/189685706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec0b0bc0 con 0x7fcfd0006130 2024-03-16T17:57:47.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.452+0000 7fcff3000640 1 -- 172.21.15.82:0/189685706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec0b0ea0 con 0x7fcfd0006130 2024-03-16T17:57:47.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.453+0000 7fcff3000640 1 -- 172.21.15.82:0/189685706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec067f30 con 0x7fcfd0006130 2024-03-16T17:57:47.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.454+0000 7fcfe3e00640 1 -- 172.21.15.82:0/189685706 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fcfdc015460 con 0x7fcfd0006130 2024-03-16T17:57:47.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.455+0000 7fcfe3e00640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfcc06b3e0 0x7fcfcc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.455+0000 7fcfe3e00640 1 -- 172.21.15.82:0/189685706 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfdc08a0b0 con 0x7fcfd0006130 2024-03-16T17:57:47.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.455+0000 7fcfeb400640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfcc06b3e0 0x7fcfcc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.456+0000 7fcfeb400640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfcc06b3e0 0x7fcfcc06d8a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800c240 tx=0x7fcfd8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:47.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.460+0000 7fcfe3e00640 1 -- 172.21.15.82:0/189685706 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcfdc054d00 con 0x7fcfd0006130 2024-03-16T17:57:47.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.659+0000 7fcff3000640 1 -- 172.21.15.82:0/189685706 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfec06d460 con 0x7fcfd0006130 2024-03-16T17:57:47.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.662+0000 7fcfe3e00640 1 -- 172.21.15.82:0/189685706 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fcfdc0870d0 con 0x7fcfd0006130 2024-03-16T17:57:47.666 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:47.666 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 -- 172.21.15.82:0/189685706 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfcc06b3e0 msgr2=0x7fcfcc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfcc06b3e0 0x7fcfcc06d8a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800c240 tx=0x7fcfd8009030 comp rx=0 tx=0).stop 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 -- 172.21.15.82:0/189685706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 msgr2=0x7fcfec0aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 0x7fcfec0aee90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc011630 tx=0x7fcfdc011af0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 -- 172.21.15.82:0/189685706 shutdown_connections 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfcc06b3e0 0x7fcfcc06d8a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfec1f7780 0x7fcfec0710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfec0b01e0 0x7fcfec1f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 --2- 172.21.15.82:0/189685706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd0006130 0x7fcfec0aee90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 -- 172.21.15.82:0/189685706 >> 172.21.15.82:0/189685706 conn(0x7fcfec06ae70 msgr2=0x7fcfec0cbee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 -- 172.21.15.82:0/189685706 shutdown_connections 2024-03-16T17:57:47.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.665+0000 7fcff3000640 1 -- 172.21.15.82:0/189685706 wait complete. 2024-03-16T17:57:47.687 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-16T17:57:47.688 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-16T17:57:47.688 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-16T17:57:47.689 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-16T17:57:47.689 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-16T17:57:47.689 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-16T17:57:47.690 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-16T17:57:47.690 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-16T17:57:47.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.978+0000 7f52ee200640 1 Processor -- start 2024-03-16T17:57:47.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.978+0000 7f52ee200640 1 -- start start 2024-03-16T17:57:47.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.980+0000 7f52ee200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52e00e3040 0x7f52e003c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.980+0000 7f52ee200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f52e00e3420 0x7f52e00e78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.980+0000 7f52ee200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e0006860 con 0x7f52e0058130 2024-03-16T17:57:47.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.980+0000 7f52ee200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e0058980 con 0x7f52e00e3040 2024-03-16T17:57:47.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.980+0000 7f52ee200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e0058c60 con 0x7f52e00e3420 2024-03-16T17:57:47.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.981+0000 7f52e7400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52e00e3040 0x7f52e003c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.981+0000 7f52e7400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52e00e3040 0x7f52e003c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35170/0 (socket says 172.21.15.82:35170) 2024-03-16T17:57:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.981+0000 7f52e7400640 1 -- 172.21.15.82:0/2595936454 learned_addr learned my addr 172.21.15.82:0/2595936454 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.981+0000 7f52e6a00640 1 --2- 172.21.15.82:0/2595936454 >> v2:172.21.15.90:3300/0 conn(0x7f52e00e3420 0x7f52e00e78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.982+0000 7f52e6a00640 1 -- 172.21.15.82:0/2595936454 >> v2:172.21.15.82:3300/0 conn(0x7f52e00e3040 msgr2=0x7f52e003c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.982+0000 7f52e6000640 1 -- 172.21.15.82:0/2595936454 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784475389 0 0) 0x7f52e0006860 con 0x7f52e0058130 2024-03-16T17:57:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.982+0000 7f52e6a00640 1 --2- 172.21.15.82:0/2595936454 >> v2:172.21.15.82:3300/0 conn(0x7f52e00e3040 0x7f52e003c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.982+0000 7f52e6a00640 1 -- 172.21.15.82:0/2595936454 >> v1:172.21.15.90:6789/0 conn(0x7f52e0058130 legacy=0x7f52e0053d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.982+0000 7f52e6a00640 1 -- 172.21.15.82:0/2595936454 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e0006210 con 0x7f52e00e3420 2024-03-16T17:57:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.982+0000 7f52e7400640 1 --2- 172.21.15.82:0/2595936454 >> v2:172.21.15.82:3300/0 conn(0x7f52e00e3040 0x7f52e003c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:47.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.985+0000 7f52e6a00640 1 --2- 172.21.15.82:0/2595936454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52e00e3420 0x7f52e00e78d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f52e003eff0 tx=0x7f52d402b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98903eb569cb6900 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:47.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.985+0000 7f52e6000640 1 -- 172.21.15.82:0/2595936454 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40317c0 con 0x7f52e00e3420 2024-03-16T17:57:47.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.986+0000 7f52e6000640 1 -- 172.21.15.82:0/2595936454 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52d4031920 con 0x7f52e00e3420 2024-03-16T17:57:47.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.986+0000 7f52e6000640 1 -- 172.21.15.82:0/2595936454 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40317c0 con 0x7f52e00e3420 2024-03-16T17:57:47.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.987+0000 7fa8cfe00640 1 Processor -- start 2024-03-16T17:57:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.987+0000 7f52ee200640 1 -- 172.21.15.82:0/2595936454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52e00e3420 msgr2=0x7f52e00e78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.987+0000 7f52ee200640 1 --2- 172.21.15.82:0/2595936454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52e00e3420 0x7f52e00e78d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f52e003eff0 tx=0x7f52d402b860 comp rx=0 tx=0).stop 2024-03-16T17:57:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.987+0000 7f52ee200640 1 -- 172.21.15.82:0/2595936454 shutdown_connections 2024-03-16T17:57:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.987+0000 7f52ee200640 1 --2- 172.21.15.82:0/2595936454 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52e00e3420 0x7f52e00e78d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.987+0000 7f52ee200640 1 --2- 172.21.15.82:0/2595936454 >> v2:172.21.15.82:3300/0 conn(0x7f52e00e3040 0x7f52e003c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:47.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.987+0000 7f52ee200640 1 -- 172.21.15.82:0/2595936454 >> 172.21.15.82:0/2595936454 conn(0x7f52e003aa40 msgr2=0x7f52e003ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:47.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.988+0000 7f52ee200640 1 -- 172.21.15.82:0/2595936454 shutdown_connections 2024-03-16T17:57:47.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.988+0000 7f52ee200640 1 -- 172.21.15.82:0/2595936454 wait complete. 2024-03-16T17:57:47.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.988+0000 7fa8cfe00640 1 -- start start 2024-03-16T17:57:47.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.990+0000 7f657a600640 1 Processor -- start 2024-03-16T17:57:47.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.990+0000 7f657a600640 1 -- start start 2024-03-16T17:57:47.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.990+0000 7fa8cfe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8d00b3520 0x7fa8d00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.990+0000 7fa8cfe00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d00d42d0 con 0x7fa8d00d0760 2024-03-16T17:57:47.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.990+0000 7fa8cfe00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d00d1250 con 0x7fa8d00cf200 2024-03-16T17:57:47.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.990+0000 7fa8cfe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d00d4520 con 0x7fa8d00b3520 2024-03-16T17:57:47.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.990+0000 7f52ee200640 1 Processor -- start 2024-03-16T17:57:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.991+0000 7f52ee200640 1 -- start start 2024-03-16T17:57:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f52ee200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0058130 0x7f52e0180b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f52ee200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52e00e3040 0x7f52e003dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f52ee200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f52e00e3420 0x7f52e003e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f52ee200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e0182070 con 0x7f52e0058130 2024-03-16T17:57:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f52ee200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e01813f0 con 0x7f52e00e3040 2024-03-16T17:57:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f52ee200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e01816d0 con 0x7f52e00e3420 2024-03-16T17:57:47.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f657a600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65741469a0 0x7f6574161860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f657a600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65740b11c0 0x7f6574161da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f657a600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6574071260 0x7f65740716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f657a600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65740ad850 con 0x7f65741469a0 2024-03-16T17:57:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f657a600640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65740725e0 con 0x7f6574071260 2024-03-16T17:57:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7f657a600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65741625e0 con 0x7f65740b11c0 2024-03-16T17:57:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.993+0000 7f6573e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65740b11c0 0x7f6574161da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.993+0000 7f6573e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65740b11c0 0x7f6574161da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:59984/0 (socket says 172.21.15.82:59984) 2024-03-16T17:57:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.993+0000 7f6573e00640 1 -- 172.21.15.82:0/2403813914 learned_addr learned my addr 172.21.15.82:0/2403813914 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.993+0000 7f6579c00640 1 --2- 172.21.15.82:0/2403813914 >> v2:172.21.15.90:3300/0 conn(0x7f6574071260 0x7f65740716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:47.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7f6579200640 1 --2- 172.21.15.82:0/2403813914 >> v2:172.21.15.82:3300/0 conn(0x7f65741469a0 0x7f6574161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.991+0000 7fa8c7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8d00b3520 0x7fa8d00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.991+0000 7fa8c7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8d00b3520 0x7fa8d00b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35184/0 (socket says 172.21.15.82:35184) 2024-03-16T17:57:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.991+0000 7fa8cea00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa8d00d0760 0x7fa8d00d0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56180/0 (socket says 172.21.15.82:56180) 2024-03-16T17:57:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.992+0000 7fa8cea00640 1 -- 172.21.15.82:0/4240060251 learned_addr learned my addr 172.21.15.82:0/4240060251 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.993+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391691163 0 0) 0x7fa8d00d42d0 con 0x7fa8d00d0760 2024-03-16T17:57:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.993+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b4003520 con 0x7fa8d00d0760 2024-03-16T17:57:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2120946192 0 0) 0x7fa8b4003520 con 0x7fa8d00d0760 2024-03-16T17:57:48.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7f52e7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0058130 0x7f52e0180b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 >> v2:172.21.15.82:3300/0 conn(0x7fa8d00b3520 msgr2=0x7fa8d00b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7fa8ce000640 1 --2- 172.21.15.82:0/4240060251 >> v2:172.21.15.82:3300/0 conn(0x7fa8d00b3520 0x7fa8d00b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 >> v1:172.21.15.90:6789/0 conn(0x7fa8d00cf200 legacy=0x7fa8d00cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7f52e6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52e00e3040 0x7f52e003dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00d2570 con 0x7fa8d00d0760 2024-03-16T17:57:48.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7f6573e00640 1 -- 172.21.15.82:0/2403813914 >> v2:172.21.15.90:3300/0 conn(0x7f6574071260 msgr2=0x7f65740716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7f6573e00640 1 --2- 172.21.15.82:0/2403813914 >> v2:172.21.15.90:3300/0 conn(0x7f6574071260 0x7f65740716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7f6573e00640 1 -- 172.21.15.82:0/2403813914 >> v2:172.21.15.82:3300/0 conn(0x7f65741469a0 msgr2=0x7f6574161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7f52e7e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f52e00e3420 0x7f52e003e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7f6573e00640 1 --2- 172.21.15.82:0/2403813914 >> v2:172.21.15.82:3300/0 conn(0x7f65741469a0 0x7f6574161860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7f6573e00640 1 -- 172.21.15.82:0/2403813914 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6574072830 con 0x7f65740b11c0 2024-03-16T17:57:48.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.994+0000 7f52e7e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f52e00e3420 0x7f52e003e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60000/0 (socket says 172.21.15.82:60000) 2024-03-16T17:57:48.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.995+0000 7f52e7e00640 1 -- 172.21.15.82:0/834130366 learned_addr learned my addr 172.21.15.82:0/834130366 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.995+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fa8c0001670 con 0x7fa8d00d0760 2024-03-16T17:57:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.995+0000 7fa8c7e00640 1 --2- 172.21.15.82:0/4240060251 >> v2:172.21.15.82:3300/0 conn(0x7fa8d00b3520 0x7fa8d00b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.995+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 >> v1:172.21.15.82:6789/0 conn(0x7fa8d00d0760 legacy=0x7fa8d00d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.996+0000 7fa8ce000640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40039c0 0x7fa8b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.996+0000 7fa8ce000640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8b40047c0 0x7fa8b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.996+0000 7fa8ce000640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b40051d0 0x7fa8b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.996+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00d4520 con 0x7fa8b40039c0 2024-03-16T17:57:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.996+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b400c120 con 0x7fa8b40051d0 2024-03-16T17:57:48.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.996+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b400c2e0 con 0x7fa8b40047c0 2024-03-16T17:57:48.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.996+0000 7f52e7400640 1 -- 172.21.15.82:0/834130366 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f52e00e3420 msgr2=0x7f52e003e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f52e7400640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f52e00e3420 0x7f52e003e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f6573e00640 1 --2- 172.21.15.82:0/2403813914 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65740b11c0 0x7f6574161da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f657406d4e0 tx=0x7f656002b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ecf30e04e0bbd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f52e7400640 1 -- 172.21.15.82:0/834130366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52e00e3040 msgr2=0x7f52e003dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f52e7400640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52e00e3040 0x7f52e003dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7fa8c7e00640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8b40047c0 0x7fa8b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7fa8cea00640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40039c0 0x7fa8b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f52e7400640 1 -- 172.21.15.82:0/834130366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc0094d0 con 0x7f52e0058130 2024-03-16T17:57:48.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7fa8cf400640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b40051d0 0x7fa8b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f88adc00640 1 Processor -- start 2024-03-16T17:57:48.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f52e7400640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0058130 0x7f52e0180b60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00e900 tx=0x7f52dc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f88adc00640 1 -- start start 2024-03-16T17:57:48.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f6573400640 1 -- 172.21.15.82:0/2403813914 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65600317f0 con 0x7f65740b11c0 2024-03-16T17:57:48.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f6573400640 1 -- 172.21.15.82:0/2403813914 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6560031950 con 0x7f65740b11c0 2024-03-16T17:57:48.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.997+0000 7f6573400640 1 -- 172.21.15.82:0/2403813914 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560031ce0 con 0x7f65740b11c0 2024-03-16T17:57:48.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7fa8c7e00640 1 -- 172.21.15.82:0/4240060251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b40051d0 msgr2=0x7fa8b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7fa8c7e00640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8b40051d0 0x7fa8b400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7f52db400640 1 -- 172.21.15.82:0/834130366 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc00ca00 con 0x7f52e0058130 2024-03-16T17:57:48.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7fa8c7e00640 1 -- 172.21.15.82:0/4240060251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40039c0 msgr2=0x7fa8b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7f52db400640 1 -- 172.21.15.82:0/834130366 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52dc00cb60 con 0x7f52e0058130 2024-03-16T17:57:48.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7f52db400640 1 -- 172.21.15.82:0/834130366 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52dc004030 con 0x7f52e0058130 2024-03-16T17:57:48.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d4009400 con 0x7f52e0058130 2024-03-16T17:57:48.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7f657a600640 1 -- 172.21.15.82:0/2403813914 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65740b11c0 msgr2=0x7f6574161da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7f657a600640 1 --2- 172.21.15.82:0/2403813914 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65740b11c0 0x7f6574161da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f657406d4e0 tx=0x7f656002b890 comp rx=0 tx=0).stop 2024-03-16T17:57:48.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.999+0000 7f657a600640 1 -- 172.21.15.82:0/2403813914 shutdown_connections 2024-03-16T17:57:48.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.999+0000 7f657a600640 1 --2- 172.21.15.82:0/2403813914 >> v2:172.21.15.90:3300/0 conn(0x7f6574071260 0x7f65740716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.999+0000 7f657a600640 1 --2- 172.21.15.82:0/2403813914 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65740b11c0 0x7f6574161da0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.999+0000 7f657a600640 1 --2- 172.21.15.82:0/2403813914 >> v2:172.21.15.82:3300/0 conn(0x7f65741469a0 0x7f6574161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e0181c90 con 0x7f52e0058130 2024-03-16T17:57:48.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.999+0000 7f657a600640 1 -- 172.21.15.82:0/2403813914 >> 172.21.15.82:0/2403813914 conn(0x7f65740af690 msgr2=0x7f65740afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.000+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52e00544a0 con 0x7f52e0058130 2024-03-16T17:57:48.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.000+0000 7f657a600640 1 -- 172.21.15.82:0/2403813914 shutdown_connections 2024-03-16T17:57:48.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.001+0000 7f657a600640 1 -- 172.21.15.82:0/2403813914 wait complete. 2024-03-16T17:57:48.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.001+0000 7f0ff9800640 1 Processor -- start 2024-03-16T17:57:48.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.999+0000 7f88adc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88a815e190 0x7f88a80d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.999+0000 7f88adc00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f88a80cf9a0 0x7f88a80d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.999+0000 7f88adc00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a80d4cb0 con 0x7f88a80b2e30 2024-03-16T17:57:48.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.999+0000 7f88adc00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a80d1f70 con 0x7f88a80cf9a0 2024-03-16T17:57:48.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.999+0000 7f88adc00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a80d49d0 con 0x7f88a815e190 2024-03-16T17:57:48.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.000+0000 7f88a6000640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f88a80b2e30 0x7f88a80b0ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.82:44734/0 (socket says 172.21.15.82:44734) 2024-03-16T17:57:48.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.000+0000 7f88a6000640 1 -- 172.21.15.82:0/2081636173 learned_addr learned my addr 172.21.15.82:0/2081636173 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:48.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.000+0000 7f88a7400640 1 --2- 172.21.15.82:0/2081636173 >> v2:172.21.15.90:3300/0 conn(0x7f88a80cf9a0 0x7f88a80d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7fa8c7e00640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40039c0 0x7fa8b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7fa8c7e00640 1 -- 172.21.15.82:0/4240060251 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b4003520 con 0x7fa8b40047c0 2024-03-16T17:57:48.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:47.998+0000 7fa8cf400640 1 -- 172.21.15.82:0/4240060251 reap_dead start 2024-03-16T17:57:48.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.000+0000 7fa8cea00640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40039c0 0x7fa8b4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:48.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.000+0000 7fa8c7e00640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8b40047c0 0x7fa8b4004c40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00b5590 tx=0x7fa8b40057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5935b274c3d8d6ed server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.001+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8030400 con 0x7fa8b40047c0 2024-03-16T17:57:48.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.001+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b8030560 con 0x7fa8b40047c0 2024-03-16T17:57:48.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.001+0000 7fa8ce000640 1 -- 172.21.15.82:0/4240060251 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b80308d0 con 0x7fa8b40047c0 2024-03-16T17:57:48.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f88a5600640 1 -- 172.21.15.82:0/2081636173 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685163307 0 0) 0x7f88a80d4cb0 con 0x7f88a80b2e30 2024-03-16T17:57:48.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f88a5600640 1 -- 172.21.15.82:0/2081636173 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f888c003520 con 0x7f88a80b2e30 2024-03-16T17:57:48.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f7bfea00640 1 Processor -- start 2024-03-16T17:57:48.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7fa8cfe00640 1 -- 172.21.15.82:0/4240060251 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8b40047c0 msgr2=0x7fa8b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7fa8cfe00640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8b40047c0 0x7fa8b4004c40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00b5590 tx=0x7fa8b40057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.001+0000 7f0ff9800640 1 -- start start 2024-03-16T17:57:48.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f88a6a00640 1 --2- 172.21.15.82:0/2081636173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88a815e190 0x7f88a80d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f88a7400640 1 -- 172.21.15.82:0/2081636173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88a815e190 msgr2=0x7f88a80d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f88a7400640 1 --2- 172.21.15.82:0/2081636173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88a815e190 0x7f88a80d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f88a7400640 1 -- 172.21.15.82:0/2081636173 >> v1:172.21.15.90:6789/0 conn(0x7f88a80b2e30 legacy=0x7f88a80b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f88a7400640 1 -- 172.21.15.82:0/2081636173 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a80d1c90 con 0x7f88a80cf9a0 2024-03-16T17:57:48.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f88a6a00640 1 --2- 172.21.15.82:0/2081636173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88a815e190 0x7f88a80d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:48.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f52db400640 1 -- 172.21.15.82:0/834130366 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f52dc004190 con 0x7f52e0058130 2024-03-16T17:57:48.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.003+0000 7f0ff9800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0ff40b31c0 0x7f0ff4145b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.003+0000 7fa8cfe00640 1 -- 172.21.15.82:0/4240060251 shutdown_connections 2024-03-16T17:57:48.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.004+0000 7fa8cfe00640 1 --2- 172.21.15.82:0/4240060251 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8b40047c0 0x7fa8b4004c40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.004+0000 7fa8cfe00640 1 -- 172.21.15.82:0/4240060251 >> 172.21.15.82:0/4240060251 conn(0x7fa8d009cf30 msgr2=0x7fa8d00d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.004+0000 7fa8cfe00640 1 -- 172.21.15.82:0/4240060251 shutdown_connections 2024-03-16T17:57:48.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.003+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ff41455c0 0x7f0ff4147a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.003+0000 7f0ff9800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff4148820 con 0x7f0ff41451e0 2024-03-16T17:57:48.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.003+0000 7f0ff9800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff4148b00 con 0x7f0ff40b31c0 2024-03-16T17:57:48.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.003+0000 7f0ff2a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0ff41451e0 0x7f0ff40b2ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56182/0 (socket says 172.21.15.82:56182) 2024-03-16T17:57:48.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.003+0000 7f0ff9800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40cdaa0 con 0x7f0ff41455c0 2024-03-16T17:57:48.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.003+0000 7f0ff2a00640 1 -- 172.21.15.82:0/3611385831 learned_addr learned my addr 172.21.15.82:0/3611385831 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:48.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.002+0000 7f7bfea00640 1 -- start start 2024-03-16T17:57:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.004+0000 7fa8cfe00640 1 -- 172.21.15.82:0/4240060251 wait complete. 2024-03-16T17:57:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.003+0000 7f52db400640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f52c806b400 0x7f52c806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.003+0000 7f52db400640 1 -- 172.21.15.82:0/834130366 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52dc08a080 con 0x7f52e0058130 2024-03-16T17:57:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.004+0000 7f52db400640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] conn(0x7f52c8073620 0x7f52c8075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.004+0000 7f52db400640 1 -- 172.21.15.82:0/834130366 --> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52c80760f0 con 0x7f52c8073620 2024-03-16T17:57:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.004+0000 7f52db400640 1 -- 172.21.15.82:0/834130366 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52dc08a3f0 con 0x7f52e0058130 2024-03-16T17:57:48.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.004+0000 7f52e7e00640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] conn(0x7f52c8073620 0x7f52c8075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.004+0000 7f88a7400640 1 --2- 172.21.15.82:0/2081636173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88a80cf9a0 0x7f88a80d26f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009d90 tx=0x7f88a002bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7593e338aa30e3c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.005+0000 7f7bfea00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7bf80b31c0 0x7f7bf8145b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.004+0000 7f0ff3400640 1 --2- 172.21.15.82:0/3611385831 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ff41455c0 0x7f0ff4147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.005+0000 7f7bfea00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf8148820 con 0x7f7bf81455c0 2024-03-16T17:57:48.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.005+0000 7f7bfea00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf80cdaa0 con 0x7f7bf81451e0 2024-03-16T17:57:48.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.005+0000 7f7bfea00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf8148b00 con 0x7f7bf80b31c0 2024-03-16T17:57:48.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.005+0000 7f52e6a00640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f52c806b400 0x7f52c806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.005+0000 7f657a600640 1 Processor -- start 2024-03-16T17:57:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.005+0000 7f657a600640 1 -- start start 2024-03-16T17:57:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.005+0000 7f0ff2000640 1 --2- 172.21.15.82:0/3611385831 >> v2:172.21.15.90:3300/0 conn(0x7f0ff40b31c0 0x7f0ff4145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7f0ff1600640 1 -- 172.21.15.82:0/3611385831 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187573613 0 0) 0x7f0ff4148820 con 0x7f0ff41451e0 2024-03-16T17:57:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7f88a5600640 1 -- 172.21.15.82:0/2081636173 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a002cb30 con 0x7f88a80cf9a0 2024-03-16T17:57:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7f88a5600640 1 -- 172.21.15.82:0/2081636173 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a002cc90 con 0x7f88a80cf9a0 2024-03-16T17:57:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7f88a5600640 1 -- 172.21.15.82:0/2081636173 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0034440 con 0x7f88a80cf9a0 2024-03-16T17:57:48.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7f7bf7e00640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f7bf81451e0 0x7f7bf80b2ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.82:44736/0 (socket says 172.21.15.82:44736) 2024-03-16T17:57:48.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7f7bf7e00640 1 -- 172.21.15.82:0/2758828015 learned_addr learned my addr 172.21.15.82:0/2758828015 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:48.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7f0ff1600640 1 -- 172.21.15.82:0/3611385831 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd8003520 con 0x7f0ff41451e0 2024-03-16T17:57:48.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7fa8cfe00640 1 Processor -- start 2024-03-16T17:57:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7f52e6a00640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f52c806b400 0x7f52c806d8c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52d4036000 tx=0x7f52d403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7fa8cfe00640 1 -- start start 2024-03-16T17:57:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7f7befe00640 1 --2- 172.21.15.82:0/2758828015 >> v2:172.21.15.90:3300/0 conn(0x7f7bf80b31c0 0x7f7bf8145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f657a600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574071260 0x7f6574144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f657a600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65740b11c0 0x7f657415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f657a600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65741469a0 0x7f6574138740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f657a600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6574145790 con 0x7f6574071260 2024-03-16T17:57:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f657a600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6574145a70 con 0x7f65740b11c0 2024-03-16T17:57:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f657a600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6574145d50 con 0x7f65741469a0 2024-03-16T17:57:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.006+0000 7f7bf7400640 1 -- 172.21.15.82:0/2758828015 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482722795 0 0) 0x7f7bf80cdaa0 con 0x7f7bf81451e0 2024-03-16T17:57:48.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f7bf7400640 1 -- 172.21.15.82:0/2758828015 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bdc003520 con 0x7f7bf81451e0 2024-03-16T17:57:48.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f88adc00640 1 -- 172.21.15.82:0/2081636173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88a80cf9a0 msgr2=0x7f88a80d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f88adc00640 1 --2- 172.21.15.82:0/2081636173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88a80cf9a0 0x7f88a80d26f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009d90 tx=0x7f88a002bd80 comp rx=0 tx=0).stop 2024-03-16T17:57:48.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f52e7e00640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] conn(0x7f52c8073620 0x7f52c8075a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f6579200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574071260 0x7f6574144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f7befe00640 1 -- 172.21.15.82:0/2758828015 >> v1:172.21.15.90:6789/0 conn(0x7f7bf81451e0 legacy=0x7f7bf80b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f7befe00640 1 -- 172.21.15.82:0/2758828015 >> v1:172.21.15.82:6789/0 conn(0x7f7bf81455c0 legacy=0x7f7bf8147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f7befe00640 1 -- 172.21.15.82:0/2758828015 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf80cdd80 con 0x7f7bf80b31c0 2024-03-16T17:57:48.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f6579c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65741469a0 0x7f6574138740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f6579200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574071260 0x7f6574144f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35224/0 (socket says 172.21.15.82:35224) 2024-03-16T17:57:48.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.007+0000 7f6579200640 1 -- 172.21.15.82:0/969034735 learned_addr learned my addr 172.21.15.82:0/969034735 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7f6573e00640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65740b11c0 0x7f657415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40047c0 0x7fa8d0152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d0152a70 0x7fa8d00c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7fa8cfe00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d006b640 0x7fa8d00c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7fa8cfe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d006be40 con 0x7fa8b40047c0 2024-03-16T17:57:48.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7fa8cfe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00c80d0 con 0x7fa8d0152a70 2024-03-16T17:57:48.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7fa8cfe00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0153420 con 0x7fa8d006b640 2024-03-16T17:57:48.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7f0ff2000640 1 -- 172.21.15.82:0/3611385831 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ff41455c0 msgr2=0x7f0ff4147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7f0ff2000640 1 --2- 172.21.15.82:0/3611385831 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ff41455c0 0x7f0ff4147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7f0ff2000640 1 -- 172.21.15.82:0/3611385831 >> v1:172.21.15.82:6789/0 conn(0x7f0ff41451e0 legacy=0x7f0ff40b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7f0ff2000640 1 -- 172.21.15.82:0/3611385831 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40cdd80 con 0x7f0ff40b31c0 2024-03-16T17:57:48.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7fa8cea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40047c0 0x7fa8d0152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7f52db400640 1 -- 172.21.15.82:0/834130366 <== osd.0 v2:172.21.15.82:6800/2810023632 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f52c80760f0 con 0x7f52c8073620 2024-03-16T17:57:48.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7f6573e00640 1 -- 172.21.15.82:0/969034735 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65741469a0 msgr2=0x7f6574138740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7f6573e00640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f65741469a0 0x7f6574138740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7f6573e00640 1 -- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574071260 msgr2=0x7f6574144f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7f6573e00640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6574071260 0x7f6574144f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.008+0000 7f6573e00640 1 -- 172.21.15.82:0/969034735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65600094d0 con 0x7f65740b11c0 2024-03-16T17:57:48.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7fa8c7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d0152a70 0x7fa8d00c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7fa8cea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40047c0 0x7fa8d0152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35236/0 (socket says 172.21.15.82:35236) 2024-03-16T17:57:48.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7fa8cea00640 1 -- 172.21.15.82:0/3277015402 learned_addr learned my addr 172.21.15.82:0/3277015402 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7f88adc00640 1 -- 172.21.15.82:0/2081636173 shutdown_connections 2024-03-16T17:57:48.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7f88adc00640 1 --2- 172.21.15.82:0/2081636173 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88a80cf9a0 0x7f88a80d26f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7f88adc00640 1 --2- 172.21.15.82:0/2081636173 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88a815e190 0x7f88a80d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7f88adc00640 1 -- 172.21.15.82:0/2081636173 >> 172.21.15.82:0/2081636173 conn(0x7f88a80d3d80 msgr2=0x7f88a80d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7fe402200640 1 Processor -- start 2024-03-16T17:57:48.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7fa8cf400640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d006b640 0x7fa8d00c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7fa8cea00640 1 -- 172.21.15.82:0/3277015402 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d006b640 msgr2=0x7fa8d00c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7fa8cea00640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d006b640 0x7fa8d00c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7fa8cea00640 1 -- 172.21.15.82:0/3277015402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d0152a70 msgr2=0x7fa8d00c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7fa8cea00640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d0152a70 0x7fa8d00c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7f88adc00640 1 -- 172.21.15.82:0/2081636173 shutdown_connections 2024-03-16T17:57:48.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7fa8cea00640 1 -- 172.21.15.82:0/3277015402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d0152f70 con 0x7fa8b40047c0 2024-03-16T17:57:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.009+0000 7f7befe00640 1 --2- 172.21.15.82:0/2758828015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bf80b31c0 0x7f7bf8145b10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7be00097a0 tx=0x7f7be002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96116863c975be62 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7f6573e00640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65740b11c0 0x7f657415a020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6574074780 tx=0x7f65600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7fe402200640 1 -- start start 2024-03-16T17:57:48.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7f88adc00640 1 -- 172.21.15.82:0/2081636173 wait complete. 2024-03-16T17:57:48.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7f7bf7400640 1 -- 172.21.15.82:0/2758828015 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0030610 con 0x7f7bf80b31c0 2024-03-16T17:57:48.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7f7bf7400640 1 -- 172.21.15.82:0/2758828015 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be0030770 con 0x7f7bf80b31c0 2024-03-16T17:57:48.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7f7bf7400640 1 -- 172.21.15.82:0/2758828015 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0030b00 con 0x7f7bf80b31c0 2024-03-16T17:57:48.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7fa8cea00640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40047c0 0x7fa8d0152530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00072c0 tx=0x7fa8c0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7fa8c6a00640 1 -- 172.21.15.82:0/3277015402 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c000fa00 con 0x7fa8b40047c0 2024-03-16T17:57:48.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7fa8c6a00640 1 -- 172.21.15.82:0/3277015402 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c000fb60 con 0x7fa8b40047c0 2024-03-16T17:57:48.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7fa8c6a00640 1 -- 172.21.15.82:0/3277015402 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00133f0 con 0x7fa8b40047c0 2024-03-16T17:57:48.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.010+0000 7f0ff2000640 1 --2- 172.21.15.82:0/3611385831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ff40b31c0 0x7f0ff4145b10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0097a0 tx=0x7f0fdc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=348d8c949fdb59a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.011+0000 7f0ff1600640 1 -- 172.21.15.82:0/3611385831 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc031830 con 0x7f0ff40b31c0 2024-03-16T17:57:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.011+0000 7f0ff1600640 1 -- 172.21.15.82:0/3611385831 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc031990 con 0x7f0ff40b31c0 2024-03-16T17:57:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.011+0000 7f0ff1600640 1 -- 172.21.15.82:0/3611385831 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc031d60 con 0x7f0ff40b31c0 2024-03-16T17:57:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.011+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d006c050 con 0x7fa8b40047c0 2024-03-16T17:57:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.011+0000 7f6570c00640 1 -- 172.21.15.82:0/969034735 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560032890 con 0x7f65740b11c0 2024-03-16T17:57:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.011+0000 7f6570c00640 1 -- 172.21.15.82:0/969034735 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65600329f0 con 0x7f65740b11c0 2024-03-16T17:57:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.011+0000 7f6570c00640 1 -- 172.21.15.82:0/969034735 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560032de0 con 0x7f65740b11c0 2024-03-16T17:57:48.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.011+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657415f370 con 0x7f65740b11c0 2024-03-16T17:57:48.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.011+0000 7f7bfea00640 1 -- 172.21.15.82:0/2758828015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bf80b31c0 msgr2=0x7f7bf8145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.012+0000 7f7bfea00640 1 --2- 172.21.15.82:0/2758828015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bf80b31c0 0x7f7bf8145b10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7be00097a0 tx=0x7f7be002b960 comp rx=0 tx=0).stop 2024-03-16T17:57:48.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.011+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657415a560 con 0x7f65740b11c0 2024-03-16T17:57:48.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.012+0000 7f0ff9800640 1 -- 172.21.15.82:0/3611385831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ff40b31c0 msgr2=0x7f0ff4145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.012+0000 7f0ff9800640 1 --2- 172.21.15.82:0/3611385831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ff40b31c0 0x7f0ff4145b10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0097a0 tx=0x7f0fdc02b8d0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.012+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d01570d0 con 0x7fa8b40047c0 2024-03-16T17:57:48.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.012+0000 7f88adc00640 1 Processor -- start 2024-03-16T17:57:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.012+0000 7f0ff9800640 1 -- 172.21.15.82:0/3611385831 shutdown_connections 2024-03-16T17:57:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.012+0000 7f0ff9800640 1 --2- 172.21.15.82:0/3611385831 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ff41455c0 0x7f0ff4147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.012+0000 7f0ff9800640 1 --2- 172.21.15.82:0/3611385831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ff40b31c0 0x7f0ff4145b10 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0097a0 tx=0x7f0fdc02b8d0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.012+0000 7f0ff9800640 1 -- 172.21.15.82:0/3611385831 >> 172.21.15.82:0/3611385831 conn(0x7f0ff40b17b0 msgr2=0x7f0ff40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.013+0000 7f7bfea00640 1 -- 172.21.15.82:0/2758828015 shutdown_connections 2024-03-16T17:57:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.013+0000 7f7bfea00640 1 --2- 172.21.15.82:0/2758828015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bf80b31c0 0x7f7bf8145b10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.013+0000 7f7bfea00640 1 -- 172.21.15.82:0/2758828015 >> 172.21.15.82:0/2758828015 conn(0x7f7bf80b17b0 msgr2=0x7f7bf80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.013+0000 7fe402200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe3fc1401f0 0x7fe3fc0b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.013+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8d00d4800 con 0x7fa8b40047c0 2024-03-16T17:57:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.013+0000 7f0ff9800640 1 -- 172.21.15.82:0/3611385831 shutdown_connections 2024-03-16T17:57:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.012+0000 7f88adc00640 1 -- start start 2024-03-16T17:57:48.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.013+0000 7f7bfea00640 1 -- 172.21.15.82:0/2758828015 shutdown_connections 2024-03-16T17:57:48.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7fe402200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1405d0 0x7fe3fc0b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7f8ccbe00640 1 Processor -- start 2024-03-16T17:57:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65740b44e0 con 0x7f65740b11c0 2024-03-16T17:57:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7fe402200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3fc0cda50 0x7fe3fc0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7f88adc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88a80b2e30 0x7f88a81f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7f7bfea00640 1 -- 172.21.15.82:0/2758828015 wait complete. 2024-03-16T17:57:48.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7f88adc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88a80cf9a0 0x7f88a8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7f88adc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88a815e190 0x7f88a80c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7fe402200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3fc0cf340 con 0x7fe3fc0cda50 2024-03-16T17:57:48.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7fe402200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3fc070940 con 0x7fe3fc1401f0 2024-03-16T17:57:48.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7fe402200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc0ce7c0 con 0x7fe3fc1405d0 2024-03-16T17:57:48.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7f88adc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a81f4c00 con 0x7f88a80b2e30 2024-03-16T17:57:48.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7f88adc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a8159680 con 0x7f88a815e190 2024-03-16T17:57:48.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7f88adc00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a81f4920 con 0x7f88a80cf9a0 2024-03-16T17:57:48.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.014+0000 7f8ccbe00640 1 -- start start 2024-03-16T17:57:48.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.015+0000 7fe3fbe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3fc0cda50 0x7fe3fc0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.015+0000 7f88a6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88a80b2e30 0x7f88a81f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.015+0000 7fe3fbe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3fc0cda50 0x7fe3fc0cdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35238/0 (socket says 172.21.15.82:35238) 2024-03-16T17:57:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.015+0000 7fe3fbe00640 1 -- 172.21.15.82:0/2723234148 learned_addr learned my addr 172.21.15.82:0/2723234148 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.015+0000 7f88a6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88a80b2e30 0x7f88a81f4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35242/0 (socket says 172.21.15.82:35242) 2024-03-16T17:57:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.015+0000 7f88a6a00640 1 -- 172.21.15.82:0/1939743973 learned_addr learned my addr 172.21.15.82:0/1939743973 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.015+0000 7f88a6000640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88a80cf9a0 0x7f88a8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.015+0000 7f88a7400640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88a815e190 0x7f88a80c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7fe3faa00640 1 --2- 172.21.15.82:0/2723234148 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1405d0 0x7fe3fc0b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7fe3fbe00640 1 -- 172.21.15.82:0/2723234148 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1405d0 msgr2=0x7fe3fc0b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.015+0000 7fe3fb400640 1 --2- 172.21.15.82:0/2723234148 >> v2:172.21.15.90:3300/0 conn(0x7fe3fc1401f0 0x7fe3fc0b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7fe3fbe00640 1 --2- 172.21.15.82:0/2723234148 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1405d0 0x7fe3fc0b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7fe3fbe00640 1 -- 172.21.15.82:0/2723234148 >> v2:172.21.15.90:3300/0 conn(0x7fe3fc1401f0 msgr2=0x7fe3fc0b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7fe3fbe00640 1 --2- 172.21.15.82:0/2723234148 >> v2:172.21.15.90:3300/0 conn(0x7fe3fc1401f0 0x7fe3fc0b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7fe3fbe00640 1 -- 172.21.15.82:0/2723234148 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc0ceaa0 con 0x7fe3fc0cda50 2024-03-16T17:57:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f8ccbe00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8cc40d28c0 0x7f8cc40ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f8ccbe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8cc40b2e30 0x7f8cc40ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7fe3fb400640 1 --2- 172.21.15.82:0/2723234148 >> v2:172.21.15.90:3300/0 conn(0x7fe3fc1401f0 0x7fe3fc0b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:48.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7fe3faa00640 1 --2- 172.21.15.82:0/2723234148 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1405d0 0x7fe3fc0b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f8ccbe00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc40d4cb0 con 0x7f8cc415e190 2024-03-16T17:57:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f8ccbe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc40d1f70 con 0x7f8cc40b2e30 2024-03-16T17:57:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f8ccbe00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc40d49d0 con 0x7f8cc40d28c0 2024-03-16T17:57:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f88a6a00640 1 -- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88a80cf9a0 msgr2=0x7f88a8159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f88a6a00640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f88a80cf9a0 0x7f88a8159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f88a6a00640 1 -- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88a815e190 msgr2=0x7f88a80c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f88a6a00640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88a815e190 0x7f88a80c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f88a6a00640 1 -- 172.21.15.82:0/1939743973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a0009a70 con 0x7f88a80b2e30 2024-03-16T17:57:48.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f7bfea00640 1 Processor -- start 2024-03-16T17:57:48.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7fa8c6a00640 1 -- 172.21.15.82:0/3277015402 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa8c0013550 con 0x7fa8b40047c0 2024-03-16T17:57:48.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f8cc9e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8cc40b2e30 0x7f8cc40ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f88a6a00640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88a80b2e30 0x7f88a81f4d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f889800c7f0 tx=0x7f889800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f8cc9e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8cc40b2e30 0x7f8cc40ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35248/0 (socket says 172.21.15.82:35248) 2024-03-16T17:57:48.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f8cc9e00640 1 -- 172.21.15.82:0/2709048311 learned_addr learned my addr 172.21.15.82:0/2709048311 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f8cc9e00640 1 -- 172.21.15.82:0/2709048311 >> v2:172.21.15.90:3300/0 conn(0x7f8cc40d28c0 msgr2=0x7f8cc40ce990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T17:57:48.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f8cc9e00640 1 --2- 172.21.15.82:0/2709048311 >> v2:172.21.15.90:3300/0 conn(0x7f8cc40d28c0 0x7f8cc40ce990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f8cc9e00640 1 -- 172.21.15.82:0/2709048311 >> v1:172.21.15.90:6789/0 conn(0x7f8cc415e190 legacy=0x7f8cc40d21b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-16T17:57:48.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f8cc9e00640 1 -- 172.21.15.82:0/2709048311 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc40d1c90 con 0x7f8cc40b2e30 2024-03-16T17:57:48.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f889ea00640 1 -- 172.21.15.82:0/1939743973 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8898007a30 con 0x7f88a80b2e30 2024-03-16T17:57:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f889ea00640 1 -- 172.21.15.82:0/1939743973 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8898007b90 con 0x7f88a80b2e30 2024-03-16T17:57:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.018+0000 7fe3fbe00640 1 --2- 172.21.15.82:0/2723234148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc0cda50 0x7fe3fc0cdec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc073390 tx=0x7fe3ec02c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f6b612b67f2569 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.013+0000 7f0ff9800640 1 -- 172.21.15.82:0/3611385831 wait complete. 2024-03-16T17:57:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.016+0000 7f0ff9800640 1 Processor -- start 2024-03-16T17:57:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.018+0000 7f0ff9800640 1 -- start start 2024-03-16T17:57:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.018+0000 7f6570c00640 1 -- 172.21.15.82:0/969034735 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f656004e450 con 0x7f65740b11c0 2024-03-16T17:57:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.019+0000 7fe3fa000640 1 -- 172.21.15.82:0/2723234148 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec034440 con 0x7fe3fc0cda50 2024-03-16T17:57:48.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.019+0000 7f8cc9e00640 1 --2- 172.21.15.82:0/2709048311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc40b2e30 0x7f8cc40ceed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8cc408d6d0 tx=0x7f8cb002b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91b353f4e1f5bdc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.019+0000 7f6570c00640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f655406b400 0x7f655406d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.019+0000 7f8cc3e00640 1 -- 172.21.15.82:0/2709048311 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0031760 con 0x7f8cc40b2e30 2024-03-16T17:57:48.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.019+0000 7f6570c00640 1 -- 172.21.15.82:0/969034735 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65600abab0 con 0x7f65740b11c0 2024-03-16T17:57:48.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.019+0000 7f8cc3e00640 1 -- 172.21.15.82:0/2709048311 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb00318c0 con 0x7f8cc40b2e30 2024-03-16T17:57:48.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7f8cc3e00640 1 -- 172.21.15.82:0/2709048311 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0031c50 con 0x7f8cc40b2e30 2024-03-16T17:57:48.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7fe3fa000640 1 -- 172.21.15.82:0/2723234148 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ec0345a0 con 0x7fe3fc0cda50 2024-03-16T17:57:48.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7fe3fa000640 1 -- 172.21.15.82:0/2723234148 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec034930 con 0x7fe3fc0cda50 2024-03-16T17:57:48.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f7bfea00640 1 -- start start 2024-03-16T17:57:48.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.018+0000 7f7bfea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bf80b31c0 0x7f7bf81f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.018+0000 7f7bfea00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bf81451e0 0x7f7bf8143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.018+0000 7f7bfea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bf81455c0 0x7f7bf8143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ff41451e0 0x7f0ff414af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ff41455c0 0x7f0ff41410a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7f6570c00640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] conn(0x7f6554073680 0x7f6554075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7f6570c00640 1 -- 172.21.15.82:0/969034735 --> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6554076150 con 0x7f6554073680 2024-03-16T17:57:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7f6570c00640 1 -- 172.21.15.82:0/969034735 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65600b02c0 con 0x7f65740b11c0 2024-03-16T17:57:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7f6579c00640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] conn(0x7f6554073680 0x7f6554075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7fa8c6a00640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b4b0 0x7fa8ac06d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7f0ff9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff41467a0 0x7f0ff41415e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.017+0000 7f889ea00640 1 -- 172.21.15.82:0/1939743973 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889800ce40 con 0x7f88a80b2e30 2024-03-16T17:57:48.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a80c7a50 con 0x7f88a80b2e30 2024-03-16T17:57:48.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2709048311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc40b2e30 msgr2=0x7f8cc40ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f8ccbe00640 1 --2- 172.21.15.82:0/2709048311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc40b2e30 0x7f8cc40ceed0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8cc408d6d0 tx=0x7f8cb002b800 comp rx=0 tx=0).stop 2024-03-16T17:57:48.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a80c8010 con 0x7f88a80b2e30 2024-03-16T17:57:48.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.020+0000 7f7bf7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bf80b31c0 0x7f7bf81f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f0ff2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ff41451e0 0x7f0ff414af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f7befe00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bf81451e0 0x7f7bf8143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f7befe00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bf81451e0 0x7f7bf8143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60090/0 (socket says 172.21.15.82:60090) 2024-03-16T17:57:48.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f0ff9800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40b0770 con 0x7f0ff41467a0 2024-03-16T17:57:48.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f0ff2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ff41451e0 0x7f0ff414af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:60334/0 (socket says 172.21.15.82:60334) 2024-03-16T17:57:48.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f0ff2a00640 1 -- 172.21.15.82:0/2965484932 learned_addr learned my addr 172.21.15.82:0/2965484932 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fe402200640 1 -- 172.21.15.82:0/2723234148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc0cda50 msgr2=0x7fe3fc0cdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fe402200640 1 --2- 172.21.15.82:0/2723234148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc0cda50 0x7fe3fc0cdec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc073390 tx=0x7fe3ec02c100 comp rx=0 tx=0).stop 2024-03-16T17:57:48.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f0ff9800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40b0980 con 0x7f0ff41451e0 2024-03-16T17:57:48.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4143360 con 0x7f0ff41455c0 2024-03-16T17:57:48.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f0ff2000640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ff41455c0 0x7f0ff41410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f6579200640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f655406b400 0x7f655406d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fe402200640 1 -- 172.21.15.82:0/2723234148 shutdown_connections 2024-03-16T17:57:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fe402200640 1 --2- 172.21.15.82:0/2723234148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc0cda50 0x7fe3fc0cdec0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fe402200640 1 --2- 172.21.15.82:0/2723234148 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1405d0 0x7fe3fc0b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fe402200640 1 --2- 172.21.15.82:0/2723234148 >> v2:172.21.15.90:3300/0 conn(0x7fe3fc1401f0 0x7fe3fc0b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fe402200640 1 -- 172.21.15.82:0/2723234148 >> 172.21.15.82:0/2723234148 conn(0x7fe3fc0b17b0 msgr2=0x7fe3fc0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f0ff3400640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff41467a0 0x7f0ff41415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f7bf7e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bf80b31c0 0x7f7bf81f8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:60320/0 (socket says 172.21.15.82:60320) 2024-03-16T17:57:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fa8c6a00640 1 -- 172.21.15.82:0/3277015402 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8c0089c00 con 0x7fa8b40047c0 2024-03-16T17:57:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fa8c6a00640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] conn(0x7fa8ac073730 0x7fa8ac075b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fa8c6a00640 1 -- 172.21.15.82:0/3277015402 --> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8ac076200 con 0x7fa8ac073730 2024-03-16T17:57:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7fa8c6a00640 1 -- 172.21.15.82:0/3277015402 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8c008e240 con 0x7fa8b40047c0 2024-03-16T17:57:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f7befe00640 1 -- 172.21.15.82:0/4054326721 learned_addr learned my addr 172.21.15.82:0/4054326721 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f7bfea00640 1 -- 172.21.15.82:0/4054326721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81f9ca0 con 0x7f7bf81455c0 2024-03-16T17:57:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f7bfea00640 1 -- 172.21.15.82:0/4054326721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81f90a0 con 0x7f7bf80b31c0 2024-03-16T17:57:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f7bfea00640 1 -- 172.21.15.82:0/4054326721 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81f9340 con 0x7f7bf81451e0 2024-03-16T17:57:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f7bfca00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bf81455c0 0x7f7bf8143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2709048311 shutdown_connections 2024-03-16T17:57:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f8ccbe00640 1 --2- 172.21.15.82:0/2709048311 >> v2:172.21.15.90:3300/0 conn(0x7f8cc40d28c0 0x7f8cc40ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f8ccbe00640 1 --2- 172.21.15.82:0/2709048311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc40b2e30 0x7f8cc40ceed0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.021+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2709048311 >> 172.21.15.82:0/2709048311 conn(0x7f8cc40d3d80 msgr2=0x7f8cc40d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2709048311 shutdown_connections 2024-03-16T17:57:48.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2709048311 wait complete. 2024-03-16T17:57:48.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f88a80d1330 con 0x7f88a80b2e30 2024-03-16T17:57:48.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7fe402200640 1 -- 172.21.15.82:0/2723234148 shutdown_connections 2024-03-16T17:57:48.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7fa8c7e00640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b4b0 0x7fa8ac06d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f0ff2000640 1 -- 172.21.15.82:0/2965484932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ff41451e0 msgr2=0x7f0ff414af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f0ff2000640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ff41451e0 0x7f0ff414af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f0ff2000640 1 -- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff41467a0 msgr2=0x7f0ff41415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7f6579c00640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] conn(0x7f6554073680 0x7f6554075aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7fa8cf400640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] conn(0x7fa8ac073730 0x7fa8ac075b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.022+0000 7fe402200640 1 -- 172.21.15.82:0/2723234148 wait complete. 2024-03-16T17:57:48.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.023+0000 7fa8c7e00640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b4b0 0x7fa8ac06d970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8036000 tx=0x7fa8b8031000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.023+0000 7f6579200640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f655406b400 0x7f655406d8c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f656c0094c0 tx=0x7f656c00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.023+0000 7fa8cf400640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] conn(0x7fa8ac073730 0x7fa8ac075b50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.024+0000 7f7bfca00640 1 -- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bf81451e0 msgr2=0x7f7bf8143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.024+0000 7f7bfca00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bf81451e0 0x7f7bf8143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.024+0000 7f0ff2000640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff41467a0 0x7f0ff41415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.024+0000 7f0ff2000640 1 -- 172.21.15.82:0/2965484932 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc009400 con 0x7f0ff41455c0 2024-03-16T17:57:48.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.024+0000 7f0ff3400640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ff41467a0 0x7f0ff41415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:48.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.024+0000 7f7bfca00640 1 -- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bf80b31c0 msgr2=0x7f7bf81f8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.024+0000 7f7bfca00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bf80b31c0 0x7f7bf81f8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.024+0000 7fa8c6a00640 1 -- 172.21.15.82:0/3277015402 <== osd.2 v2:172.21.15.82:6824/1740405378 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa8ac076200 con 0x7fa8ac073730 2024-03-16T17:57:48.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.025+0000 7f7bfca00640 1 -- 172.21.15.82:0/4054326721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be0009400 con 0x7f7bf81455c0 2024-03-16T17:57:48.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.025+0000 7f8ccbe00640 1 Processor -- start 2024-03-16T17:57:48.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.025+0000 7f6570c00640 1 -- 172.21.15.82:0/969034735 <== osd.1 v2:172.21.15.82:6808/1061186346 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6554076150 con 0x7f6554073680 2024-03-16T17:57:48.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.025+0000 7f889ea00640 1 -- 172.21.15.82:0/1939743973 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f88980113f0 con 0x7f88a80b2e30 2024-03-16T17:57:48.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.025+0000 7f7bf7e00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bf80b31c0 0x7f7bf81f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:48.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.025+0000 7f8ccbe00640 1 -- start start 2024-03-16T17:57:48.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.025+0000 7f7befe00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7bf81451e0 0x7f7bf8143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:48.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.025+0000 7f0ff2000640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ff41455c0 0x7f0ff41410a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc033000 tx=0x7f0fdc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.025+0000 7fe402200640 1 Processor -- start 2024-03-16T17:57:48.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.026+0000 7fe402200640 1 -- start start 2024-03-16T17:57:48.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.026+0000 7f7bfca00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bf81455c0 0x7f7bf8143b90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c700 tx=0x7f7bf000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.026+0000 7f889ea00640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f888806b400 0x7f888806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.026+0000 7f0feaa00640 1 -- 172.21.15.82:0/2965484932 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc004030 con 0x7f0ff41455c0 2024-03-16T17:57:48.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.026+0000 7f0feaa00640 1 -- 172.21.15.82:0/2965484932 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc004190 con 0x7f0ff41455c0 2024-03-16T17:57:48.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.026+0000 7f0feaa00640 1 -- 172.21.15.82:0/2965484932 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc004500 con 0x7f0ff41455c0 2024-03-16T17:57:48.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.026+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff4143640 con 0x7f0ff41455c0 2024-03-16T17:57:48.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.027+0000 7f7bf4c00640 1 -- 172.21.15.82:0/4054326721 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0003b70 con 0x7f7bf81455c0 2024-03-16T17:57:48.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.027+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc40b2e30 0x7f8cc41f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.027+0000 7f7bf4c00640 1 -- 172.21.15.82:0/4054326721 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf0003cd0 con 0x7f7bf81455c0 2024-03-16T17:57:48.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.027+0000 7f88a6000640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f888806b400 0x7f888806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.027+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8cc40d28c0 0x7f8cc40c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.027+0000 7f7bfea00640 1 -- 172.21.15.82:0/4054326721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf81f95e0 con 0x7f7bf81455c0 2024-03-16T17:57:48.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.027+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cc415e190 0x7f8cc40c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f889ea00640 1 -- 172.21.15.82:0/1939743973 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f889808a210 con 0x7f88a80b2e30 2024-03-16T17:57:48.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7fe402200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3fc0cda50 0x7fe3fc13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff4144d70 con 0x7f0ff41455c0 2024-03-16T17:57:48.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f8cc9400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc40b2e30 0x7f8cc41f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7fe402200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1401f0 0x7fe3fc15dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f889ea00640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] conn(0x7f8888073620 0x7f8888075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f889ea00640 1 -- 172.21.15.82:0/1939743973 --> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88880760f0 con 0x7f8888073620 2024-03-16T17:57:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f889ea00640 1 -- 172.21.15.82:0/1939743973 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f889808a6b0 con 0x7f88a80b2e30 2024-03-16T17:57:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7fe402200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc1405d0 0x7fe3fc15e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7fe402200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc15e770 con 0x7fe3fc1405d0 2024-03-16T17:57:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7fe402200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc06dce0 con 0x7fe3fc0cda50 2024-03-16T17:57:48.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7fe402200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc06dfc0 con 0x7fe3fc1401f0 2024-03-16T17:57:48.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f8ccbe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc40c7870 con 0x7f8cc40b2e30 2024-03-16T17:57:48.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f8ccbe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc41f3c20 con 0x7f8cc415e190 2024-03-16T17:57:48.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f8ccbe00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc4160830 con 0x7f8cc40d28c0 2024-03-16T17:57:48.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f7bf4c00640 1 -- 172.21.15.82:0/4054326721 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0005310 con 0x7f7bf81455c0 2024-03-16T17:57:48.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f8cc9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cc415e190 0x7f8cc40c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7fe3fb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3fc0cda50 0x7fe3fc13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f8cc9e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cc415e190 0x7f8cc40c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:60342/0 (socket says 172.21.15.82:60342) 2024-03-16T17:57:48.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7fe3faa00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1401f0 0x7fe3fc15dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.028+0000 7f8cc9e00640 1 -- 172.21.15.82:0/2014774275 learned_addr learned my addr 172.21.15.82:0/2014774275 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.029+0000 7f7bfea00640 1 -- 172.21.15.82:0/4054326721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf8072b90 con 0x7f7bf81455c0 2024-03-16T17:57:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.029+0000 7f88a6000640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f888806b400 0x7f888806d8c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88a80b6780 tx=0x7f8894007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.029+0000 7fe3faa00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1401f0 0x7fe3fc15dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60124/0 (socket says 172.21.15.82:60124) 2024-03-16T17:57:48.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.029+0000 7fe3faa00640 1 -- 172.21.15.82:0/3318379428 learned_addr learned my addr 172.21.15.82:0/3318379428 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.030+0000 7f88a7400640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] conn(0x7f8888073620 0x7f8888075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.029+0000 7fe3faa00640 1 -- 172.21.15.82:0/3318379428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3fc0cda50 msgr2=0x7fe3fc13ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.030+0000 7fe3faa00640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3fc0cda50 0x7fe3fc13ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.030+0000 7fe3faa00640 1 -- 172.21.15.82:0/3318379428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc1405d0 msgr2=0x7fe3fc15e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.030+0000 7fe3faa00640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc1405d0 0x7fe3fc15e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.030+0000 7fe3faa00640 1 -- 172.21.15.82:0/3318379428 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0094d0 con 0x7fe3fc1401f0 2024-03-16T17:57:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.030+0000 7f8cc8a00640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8cc40d28c0 0x7f8cc40c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.030+0000 7fe3faa00640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1401f0 0x7fe3fc15dcf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800ce30 tx=0x7fe3e800a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ff4073420 con 0x7f0ff41455c0 2024-03-16T17:57:48.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7f7bed600640 1 -- 172.21.15.82:0/4054326721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7bf80b3c00 con 0x7f7bf81455c0 2024-03-16T17:57:48.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7fe3f3400640 1 -- 172.21.15.82:0/3318379428 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800f680 con 0x7fe3fc1401f0 2024-03-16T17:57:48.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7f88a7400640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] conn(0x7f8888073620 0x7f8888075a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7fe3f3400640 1 -- 172.21.15.82:0/3318379428 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e800f7e0 con 0x7fe3fc1401f0 2024-03-16T17:57:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7fe3f3400640 1 -- 172.21.15.82:0/3318379428 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800fbb0 con 0x7fe3fc1401f0 2024-03-16T17:57:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc06e2a0 con 0x7fe3fc1401f0 2024-03-16T17:57:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc13b390 con 0x7fe3fc1401f0 2024-03-16T17:57:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7f8cc9400640 1 -- 172.21.15.82:0/2014774275 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8cc40d28c0 msgr2=0x7f8cc40c9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7f8cc9400640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8cc40d28c0 0x7f8cc40c9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7f8cc9400640 1 -- 172.21.15.82:0/2014774275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cc415e190 msgr2=0x7f8cc40c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7f8cc9400640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cc415e190 0x7f8cc40c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.031+0000 7f8cc9400640 1 -- 172.21.15.82:0/2014774275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb00094d0 con 0x7f8cc40b2e30 2024-03-16T17:57:48.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.033+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe3d4001030 con 0x7fe3fc1401f0 2024-03-16T17:57:48.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.033+0000 7f8cc9400640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc40b2e30 0x7f8cc41f36e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00be20 tx=0x7f8cbc009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.033+0000 7f8cc1600640 1 -- 172.21.15.82:0/2014774275 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc014440 con 0x7f8cc40b2e30 2024-03-16T17:57:48.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.033+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc4160b10 con 0x7f8cc40b2e30 2024-03-16T17:57:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.033+0000 7f889ea00640 1 -- 172.21.15.82:0/1939743973 <== osd.4 v2:172.21.15.90:6800/3078143209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f88880760f0 con 0x7f8888073620 2024-03-16T17:57:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.034+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc41610d0 con 0x7f8cc40b2e30 2024-03-16T17:57:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.034+0000 7f8cc1600640 1 -- 172.21.15.82:0/2014774275 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cbc0145a0 con 0x7f8cc40b2e30 2024-03-16T17:57:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.034+0000 7f8cc1600640 1 -- 172.21.15.82:0/2014774275 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc014990 con 0x7f8cc40b2e30 2024-03-16T17:57:48.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.035+0000 7f7bf4c00640 1 -- 172.21.15.82:0/4054326721 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f7bf0005470 con 0x7f7bf81455c0 2024-03-16T17:57:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.035+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8cc415e570 con 0x7f8cc40b2e30 2024-03-16T17:57:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.036+0000 7f7bf4c00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7bd406b4b0 0x7f7bd406d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.035+0000 7fe3f3400640 1 -- 172.21.15.82:0/3318379428 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe3e800b040 con 0x7fe3fc1401f0 2024-03-16T17:57:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.037+0000 7f7bf7e00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7bd406b4b0 0x7f7bd406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.037+0000 7fe3f3400640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3dc06b3b0 0x7fe3dc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.037+0000 7f7bf4c00640 1 -- 172.21.15.82:0/4054326721 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bf0089870 con 0x7f7bf81455c0 2024-03-16T17:57:48.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.037+0000 7fe3f3400640 1 -- 172.21.15.82:0/3318379428 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3e808abb0 con 0x7fe3fc1401f0 2024-03-16T17:57:48.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.038+0000 7fe3f3400640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] conn(0x7fe3dc073550 0x7fe3dc075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.038+0000 7fe3f3400640 1 -- 172.21.15.82:0/3318379428 --> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3dc076020 con 0x7fe3dc073550 2024-03-16T17:57:48.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.038+0000 7fe3f3400640 1 -- 172.21.15.82:0/3318379428 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3e808af20 con 0x7fe3fc1401f0 2024-03-16T17:57:48.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.038+0000 7fe3fb400640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3dc06b3b0 0x7fe3dc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.036+0000 7f0feaa00640 1 -- 172.21.15.82:0/2965484932 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f0fdc003610 con 0x7f0ff41455c0 2024-03-16T17:57:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.037+0000 7f0feaa00640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0fd006b3b0 0x7f0fd006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.038+0000 7f0feaa00640 1 -- 172.21.15.82:0/2965484932 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fdc0aafb0 con 0x7f0ff41455c0 2024-03-16T17:57:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.038+0000 7f0feaa00640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] conn(0x7f0fd00735d0 0x7f0fd00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.038+0000 7f0feaa00640 1 -- 172.21.15.82:0/2965484932 --> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0fd00760a0 con 0x7f0fd00735d0 2024-03-16T17:57:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.038+0000 7f0feaa00640 1 -- 172.21.15.82:0/2965484932 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fdc0ad570 con 0x7f0ff41455c0 2024-03-16T17:57:48.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.038+0000 7f0ff2a00640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0fd006b3b0 0x7f0fd006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.038+0000 7fe3fbe00640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] conn(0x7fe3dc073550 0x7fe3dc075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.037+0000 7f7bf4c00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] conn(0x7f7bd40736d0 0x7f7bd4075af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.037+0000 7f8cc1600640 1 -- 172.21.15.82:0/2014774275 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8cbc01d3b0 con 0x7f8cc40b2e30 2024-03-16T17:57:48.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.039+0000 7f8cc1600640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8ca406b400 0x7f8ca406d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.039+0000 7f8cc1600640 1 -- 172.21.15.82:0/2014774275 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8cbc0588a0 con 0x7f8cc40b2e30 2024-03-16T17:57:48.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.039+0000 7f8cc8a00640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8ca406b400 0x7f8ca406d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.039+0000 7fe3fb400640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3dc06b3b0 0x7fe3dc06d870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400ab60 tx=0x7fe3f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.039+0000 7f8cc1600640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] conn(0x7f8ca4073680 0x7f8ca4075aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.039+0000 7f8cc1600640 1 -- 172.21.15.82:0/2014774275 --> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ca4076150 con 0x7f8ca4073680 2024-03-16T17:57:48.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.039+0000 7f8cc1600640 1 -- 172.21.15.82:0/2014774275 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cbc009890 con 0x7f8cc40b2e30 2024-03-16T17:57:48.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.039+0000 7f7bf4c00640 1 -- 172.21.15.82:0/4054326721 --> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7bd40761a0 con 0x7f7bd40736d0 2024-03-16T17:57:48.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.039+0000 7f7bf4c00640 1 -- 172.21.15.82:0/4054326721 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bf008e2a0 con 0x7f7bf81455c0 2024-03-16T17:57:48.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.039+0000 7fe3fbe00640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] conn(0x7fe3dc073550 0x7fe3dc075970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.040+0000 7f8cc9e00640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] conn(0x7f8ca4073680 0x7f8ca4075aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.040+0000 7f0ff2a00640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0fd006b3b0 0x7f0fd006d870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0fec005880 tx=0x7f0fec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.040+0000 7f7befe00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] conn(0x7f7bd40736d0 0x7f7bd4075af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.041+0000 7f0ff3400640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] conn(0x7f0fd00735d0 0x7f0fd00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.041+0000 7f7bf7e00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7bd406b4b0 0x7f7bd406d970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7be80011e0 tx=0x7f7be8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.041+0000 7f8cc8a00640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8ca406b400 0x7f8ca406d8c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8006fd0 tx=0x7f8cb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.041+0000 7f8cc9e00640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] conn(0x7f8ca4073680 0x7f8ca4075aa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.041+0000 7f0ff3400640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] conn(0x7f0fd00735d0 0x7f0fd00759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.042+0000 7fe3f3400640 1 -- 172.21.15.82:0/3318379428 <== osd.6 v2:172.21.15.90:6816/1201371759 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe3dc076020 con 0x7fe3dc073550 2024-03-16T17:57:48.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.043+0000 7f0feaa00640 1 -- 172.21.15.82:0/2965484932 <== osd.3 v2:172.21.15.82:6816/4087215098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0fd00760a0 con 0x7f0fd00735d0 2024-03-16T17:57:48.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.043+0000 7f8cc1600640 1 -- 172.21.15.82:0/2014774275 <== osd.7 v2:172.21.15.90:6817/3392471023 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8ca4076150 con 0x7f8ca4073680 2024-03-16T17:57:48.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.044+0000 7f7befe00640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] conn(0x7f7bd40736d0 0x7f7bd4075af0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.046+0000 7f7bf4c00640 1 -- 172.21.15.82:0/4054326721 <== osd.5 v2:172.21.15.90:6808/2407665501 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7bd40761a0 con 0x7f7bd40736d0 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.048+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 --> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa8d00c8590 con 0x7fa8ac073730 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.048+0000 7fa8c6a00640 1 -- 172.21.15.82:0/3277015402 <== osd.2 v2:172.21.15.82:6824/1740405378 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa8d00c8590 con 0x7fa8ac073730 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.049+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] conn(0x7fa8ac073730 msgr2=0x7fa8ac075b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.049+0000 7fa8cfe00640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] conn(0x7fa8ac073730 0x7fa8ac075b50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.049+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b4b0 msgr2=0x7fa8ac06d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.049+0000 7fa8cfe00640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b4b0 0x7fa8ac06d970 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8036000 tx=0x7fa8b8031000 comp rx=0 tx=0).stop 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.049+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40047c0 msgr2=0x7fa8d0152530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.049+0000 7fa8cfe00640 1 --2- 172.21.15.82:0/3277015402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8b40047c0 0x7fa8d0152530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00072c0 tx=0x7fa8c0007780 comp rx=0 tx=0).stop 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.049+0000 7fa8cf400640 1 -- 172.21.15.82:0/3277015402 reap_dead start 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.049+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 shutdown_connections 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.049+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 >> 172.21.15.82:0/3277015402 conn(0x7fa8d009cf30 msgr2=0x7fa8d00cf610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.052+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 shutdown_connections 2024-03-16T17:57:48.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.052+0000 7fa8cfe00640 1 -- 172.21.15.82:0/3277015402 wait complete. 2024-03-16T17:57:48.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.052+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 --> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52e003c180 con 0x7f52c8073620 2024-03-16T17:57:48.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.054+0000 7f52db400640 1 -- 172.21.15.82:0/834130366 <== osd.0 v2:172.21.15.82:6800/2810023632 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f52e003c180 con 0x7f52c8073620 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.057+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] conn(0x7f52c8073620 msgr2=0x7f52c8075a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.057+0000 7f52ee200640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] conn(0x7f52c8073620 0x7f52c8075a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.057+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f52c806b400 msgr2=0x7f52c806d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.057+0000 7f52ee200640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f52c806b400 0x7f52c806d8c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52d4036000 tx=0x7f52d403d040 comp rx=0 tx=0).stop 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.057+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0058130 msgr2=0x7f52e0180b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.057+0000 7f52ee200640 1 --2- 172.21.15.82:0/834130366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52e0058130 0x7f52e0180b60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00e900 tx=0x7f52dc00edc0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.057+0000 7f52e7e00640 1 -- 172.21.15.82:0/834130366 reap_dead start 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.058+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 shutdown_connections 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.058+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 >> 172.21.15.82:0/834130366 conn(0x7f52e003aa40 msgr2=0x7f52e003a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.058+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 shutdown_connections 2024-03-16T17:57:48.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.058+0000 7f52ee200640 1 -- 172.21.15.82:0/834130366 wait complete. 2024-03-16T17:57:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.057+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 --> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f88a815e570 con 0x7f8888073620 2024-03-16T17:57:48.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.059+0000 7f889ea00640 1 -- 172.21.15.82:0/1939743973 <== osd.4 v2:172.21.15.90:6800/3078143209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f88a002ca90 con 0x7f8888073620 2024-03-16T17:57:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.060+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] conn(0x7f8888073620 msgr2=0x7f8888075a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.060+0000 7f88adc00640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] conn(0x7f8888073620 0x7f8888075a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.060+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f888806b400 msgr2=0x7f888806d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.060+0000 7f88adc00640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f888806b400 0x7f888806d8c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f88a80b6780 tx=0x7f8894007d30 comp rx=0 tx=0).stop 2024-03-16T17:57:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.060+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88a80b2e30 msgr2=0x7f88a81f4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.060+0000 7f88adc00640 1 --2- 172.21.15.82:0/1939743973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88a80b2e30 0x7f88a81f4d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f889800c7f0 tx=0x7f889800cbf0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.060+0000 7f88a7400640 1 -- 172.21.15.82:0/1939743973 reap_dead start 2024-03-16T17:57:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.060+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 shutdown_connections 2024-03-16T17:57:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.060+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 >> 172.21.15.82:0/1939743973 conn(0x7f88a80d3d80 msgr2=0x7f88a80d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.061+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 shutdown_connections 2024-03-16T17:57:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.062+0000 7f88adc00640 1 -- 172.21.15.82:0/1939743973 wait complete. 2024-03-16T17:57:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.066+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 --> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6574161490 con 0x7f6554073680 2024-03-16T17:57:48.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.066+0000 7f6570c00640 1 -- 172.21.15.82:0/969034735 <== osd.1 v2:172.21.15.82:6808/1061186346 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6574161490 con 0x7f6554073680 2024-03-16T17:57:48.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.070+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 --> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8cc40a1db0 con 0x7f8ca4073680 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.070+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] conn(0x7f6554073680 msgr2=0x7f6554075aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.070+0000 7f657a600640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] conn(0x7f6554073680 0x7f6554075aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.070+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f655406b400 msgr2=0x7f655406d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.070+0000 7f657a600640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f655406b400 0x7f655406d8c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f656c0094c0 tx=0x7f656c00b040 comp rx=0 tx=0).stop 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.070+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65740b11c0 msgr2=0x7f657415a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.070+0000 7f657a600640 1 --2- 172.21.15.82:0/969034735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f65740b11c0 0x7f657415a020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6574074780 tx=0x7f65600046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.071+0000 7f6579c00640 1 -- 172.21.15.82:0/969034735 reap_dead start 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.071+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 shutdown_connections 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.071+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 >> 172.21.15.82:0/969034735 conn(0x7f65740af690 msgr2=0x7f65740b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.071+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 shutdown_connections 2024-03-16T17:57:48.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.071+0000 7f657a600640 1 -- 172.21.15.82:0/969034735 wait complete. 2024-03-16T17:57:48.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.072+0000 7f8cc1600640 1 -- 172.21.15.82:0/2014774275 <== osd.7 v2:172.21.15.90:6817/3392471023 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8cc40a1db0 con 0x7f8ca4073680 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.073+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 >> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] conn(0x7f8ca4073680 msgr2=0x7f8ca4075aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.074+0000 7f8ccbe00640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] conn(0x7f8ca4073680 0x7f8ca4075aa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.074+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8ca406b400 msgr2=0x7f8ca406d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.074+0000 7f8ccbe00640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8ca406b400 0x7f8ca406d8c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8006fd0 tx=0x7f8cb8008040 comp rx=0 tx=0).stop 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.074+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc40b2e30 msgr2=0x7f8cc41f36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.074+0000 7f8ccbe00640 1 --2- 172.21.15.82:0/2014774275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cc40b2e30 0x7f8cc41f36e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc00be20 tx=0x7f8cbc009f70 comp rx=0 tx=0).stop 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.074+0000 7f8cc9e00640 1 -- 172.21.15.82:0/2014774275 reap_dead start 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.074+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 shutdown_connections 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.074+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 >> 172.21.15.82:0/2014774275 conn(0x7f8cc40d3d80 msgr2=0x7f8cc406c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.075+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 shutdown_connections 2024-03-16T17:57:48.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.075+0000 7f8ccbe00640 1 -- 172.21.15.82:0/2014774275 wait complete. 2024-03-16T17:57:48.118 INFO:teuthology.orchestra.run.smithi082.stdout:60129542148 2024-03-16T17:57:48.119 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-16T17:57:48.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.077+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 --> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ff414ac40 con 0x7f0fd00735d0 2024-03-16T17:57:48.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.077+0000 7f0feaa00640 1 -- 172.21.15.82:0/2965484932 <== osd.3 v2:172.21.15.82:6816/4087215098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ff414ac40 con 0x7f0fd00735d0 2024-03-16T17:57:48.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.078+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 --> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe3d4002d30 con 0x7fe3dc073550 2024-03-16T17:57:48.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.078+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] conn(0x7f0fd00735d0 msgr2=0x7f0fd00759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.078+0000 7f0ff9800640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] conn(0x7f0fd00735d0 0x7f0fd00759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.078+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0fd006b3b0 msgr2=0x7f0fd006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.078+0000 7f0ff9800640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0fd006b3b0 0x7f0fd006d870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0fec005880 tx=0x7f0fec009040 comp rx=0 tx=0).stop 2024-03-16T17:57:48.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.078+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ff41455c0 msgr2=0x7f0ff41410a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.078+0000 7f0ff9800640 1 --2- 172.21.15.82:0/2965484932 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0ff41455c0 0x7f0ff41410a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc033000 tx=0x7f0fdc0046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.078+0000 7f7bed600640 1 -- 172.21.15.82:0/4054326721 --> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7bf814b570 con 0x7f7bd40736d0 2024-03-16T17:57:48.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.078+0000 7f0ff3400640 1 -- 172.21.15.82:0/2965484932 reap_dead start 2024-03-16T17:57:48.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.079+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 shutdown_connections 2024-03-16T17:57:48.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.079+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 >> 172.21.15.82:0/2965484932 conn(0x7f0ff40b17b0 msgr2=0x7f0ff4146090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.079+0000 7fe3f3400640 1 -- 172.21.15.82:0/3318379428 <== osd.6 v2:172.21.15.90:6816/1201371759 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe3d4002d30 con 0x7fe3dc073550 2024-03-16T17:57:48.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.079+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 shutdown_connections 2024-03-16T17:57:48.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.079+0000 7f0ff9800640 1 -- 172.21.15.82:0/2965484932 wait complete. 2024-03-16T17:57:48.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.079+0000 7f7bf4c00640 1 -- 172.21.15.82:0/4054326721 <== osd.5 v2:172.21.15.90:6808/2407665501 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7bf814b570 con 0x7f7bd40736d0 2024-03-16T17:57:48.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 >> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] conn(0x7fe3dc073550 msgr2=0x7fe3dc075970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7fe402200640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] conn(0x7fe3dc073550 0x7fe3dc075970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7f7bed600640 1 -- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] conn(0x7f7bd40736d0 msgr2=0x7f7bd4075af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7f7bed600640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] conn(0x7f7bd40736d0 0x7f7bd4075af0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7f7bed600640 1 -- 172.21.15.82:0/4054326721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7bd406b4b0 msgr2=0x7f7bd406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7f7bed600640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7bd406b4b0 0x7f7bd406d970 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7be80011e0 tx=0x7f7be8009040 comp rx=0 tx=0).stop 2024-03-16T17:57:48.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7f7bed600640 1 -- 172.21.15.82:0/4054326721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bf81455c0 msgr2=0x7f7bf8143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7f7bed600640 1 --2- 172.21.15.82:0/4054326721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7bf81455c0 0x7f7bf8143b90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c700 tx=0x7f7bf000cbc0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3dc06b3b0 msgr2=0x7fe3dc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7fe402200640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3dc06b3b0 0x7fe3dc06d870 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400ab60 tx=0x7fe3f4008040 comp rx=0 tx=0).stop 2024-03-16T17:57:48.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1401f0 msgr2=0x7fe3fc15dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7fe402200640 1 --2- 172.21.15.82:0/3318379428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe3fc1401f0 0x7fe3fc15dcf0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800ce30 tx=0x7fe3e800a650 comp rx=0 tx=0).stop 2024-03-16T17:57:48.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7fe3fbe00640 1 -- 172.21.15.82:0/3318379428 reap_dead start 2024-03-16T17:57:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 shutdown_connections 2024-03-16T17:57:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.080+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 >> 172.21.15.82:0/3318379428 conn(0x7fe3fc0b17b0 msgr2=0x7fe3fc070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.081+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 shutdown_connections 2024-03-16T17:57:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.081+0000 7fe402200640 1 -- 172.21.15.82:0/3318379428 wait complete. 2024-03-16T17:57:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.082+0000 7f7bfca00640 1 -- 172.21.15.82:0/4054326721 reap_dead start 2024-03-16T17:57:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.082+0000 7f7bed600640 1 -- 172.21.15.82:0/4054326721 shutdown_connections 2024-03-16T17:57:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.082+0000 7f7bed600640 1 -- 172.21.15.82:0/4054326721 >> 172.21.15.82:0/4054326721 conn(0x7f7bf80b17b0 msgr2=0x7f7bf8146b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.082+0000 7f7bed600640 1 -- 172.21.15.82:0/4054326721 shutdown_connections 2024-03-16T17:57:48.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.082+0000 7f7bed600640 1 -- 172.21.15.82:0/4054326721 wait complete. 2024-03-16T17:57:48.126 INFO:teuthology.orchestra.run.smithi082.stdout:60129542148 2024-03-16T17:57:48.126 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-16T17:57:48.127 INFO:teuthology.orchestra.run.smithi082.stdout:60129542150 2024-03-16T17:57:48.127 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-16T17:57:48.128 INFO:teuthology.orchestra.run.smithi082.stdout:60129542149 2024-03-16T17:57:48.128 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-16T17:57:48.129 INFO:teuthology.orchestra.run.smithi082.stdout:60129542150 2024-03-16T17:57:48.129 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-16T17:57:48.130 INFO:teuthology.orchestra.run.smithi082.stdout:60129542149 2024-03-16T17:57:48.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-16T17:57:48.131 INFO:teuthology.orchestra.run.smithi082.stdout:60129542150 2024-03-16T17:57:48.132 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-16T17:57:48.133 INFO:teuthology.orchestra.run.smithi082.stdout:60129542148 2024-03-16T17:57:48.133 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-16T17:57:48.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.458+0000 7fe83f400640 1 Processor -- start 2024-03-16T17:57:48.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.458+0000 7fe83f400640 1 -- start start 2024-03-16T17:57:48.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.460+0000 7fe83f400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe8381451e0 0x7fe838145b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.460+0000 7fe83f400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8381455c0 0x7fe8380b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.460+0000 7fe83f400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe838148820 con 0x7fe8380b2ff0 2024-03-16T17:57:48.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.461+0000 7fe83f400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe838148b00 con 0x7fe8381451e0 2024-03-16T17:57:48.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.461+0000 7fe83f400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380b3480 con 0x7fe8381455c0 2024-03-16T17:57:48.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.461+0000 7fe83ca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe8381451e0 0x7fe838145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.461+0000 7fe83ca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe8381451e0 0x7fe838145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35268/0 (socket says 172.21.15.82:35268) 2024-03-16T17:57:48.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.461+0000 7fe83ca00640 1 -- 172.21.15.82:0/2532364013 learned_addr learned my addr 172.21.15.82:0/2532364013 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.462+0000 7fe83ca00640 1 -- 172.21.15.82:0/2532364013 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8381455c0 msgr2=0x7fe8380b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.462+0000 7fe83ca00640 1 --2- 172.21.15.82:0/2532364013 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8381455c0 0x7fe8380b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.462+0000 7fe837400640 1 -- 172.21.15.82:0/2532364013 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117605890 0 0) 0x7fe838148820 con 0x7fe8380b2ff0 2024-03-16T17:57:48.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.462+0000 7fe83ca00640 1 -- 172.21.15.82:0/2532364013 >> v1:172.21.15.82:6789/0 conn(0x7fe8380b2ff0 legacy=0x7fe838147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.462+0000 7fe83ca00640 1 -- 172.21.15.82:0/2532364013 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8380cdaa0 con 0x7fe8381451e0 2024-03-16T17:57:48.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.464+0000 7fe83ca00640 1 --2- 172.21.15.82:0/2532364013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 0x7fe838145b10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe830009e50 tx=0x7fe83002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694ed8dac9c432ec server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.465+0000 7fe837400640 1 -- 172.21.15.82:0/2532364013 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83002cb00 con 0x7fe8381451e0 2024-03-16T17:57:48.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.465+0000 7fe837400640 1 -- 172.21.15.82:0/2532364013 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83002cc60 con 0x7fe8381451e0 2024-03-16T17:57:48.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.465+0000 7fe837400640 1 -- 172.21.15.82:0/2532364013 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300343f0 con 0x7fe8381451e0 2024-03-16T17:57:48.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.466+0000 7fe83f400640 1 -- 172.21.15.82:0/2532364013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 msgr2=0x7fe838145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.466+0000 7fe83f400640 1 --2- 172.21.15.82:0/2532364013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 0x7fe838145b10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe830009e50 tx=0x7fe83002bf60 comp rx=0 tx=0).stop 2024-03-16T17:57:48.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.467+0000 7fe83f400640 1 -- 172.21.15.82:0/2532364013 shutdown_connections 2024-03-16T17:57:48.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.468+0000 7fd1fc800640 1 Processor -- start 2024-03-16T17:57:48.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.468+0000 7fd1fc800640 1 -- start start 2024-03-16T17:57:48.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.468+0000 7fe83f400640 1 --2- 172.21.15.82:0/2532364013 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8381455c0 0x7fe8380b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.468+0000 7fe83f400640 1 --2- 172.21.15.82:0/2532364013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 0x7fe838145b10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.468+0000 7fe83f400640 1 -- 172.21.15.82:0/2532364013 >> 172.21.15.82:0/2532364013 conn(0x7fe8380b17b0 msgr2=0x7fe8380b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.468+0000 7fe83f400640 1 -- 172.21.15.82:0/2532364013 shutdown_connections 2024-03-16T17:57:48.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.468+0000 7f3bcb200640 1 Processor -- start 2024-03-16T17:57:48.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.468+0000 7fe83f400640 1 -- 172.21.15.82:0/2532364013 wait complete. 2024-03-16T17:57:48.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.469+0000 7f3bcb200640 1 -- start start 2024-03-16T17:57:48.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.470+0000 7fd1fc800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1f40ad850 0x7fd1f40cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.470+0000 7fd1fc800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd1f40ccc50 0x7fd1f40b3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.470+0000 7fd1fc800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1f40cd080 con 0x7fd1f41613b0 2024-03-16T17:57:48.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.470+0000 7fd1fc800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1f40ade10 con 0x7fd1f40ccc50 2024-03-16T17:57:48.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.470+0000 7fd1f9e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd1f41613b0 0x7fd1f40b47f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56202/0 (socket says 172.21.15.82:56202) 2024-03-16T17:57:48.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.470+0000 7fd1f9e00640 1 -- 172.21.15.82:0/2127157209 learned_addr learned my addr 172.21.15.82:0/2127157209 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:48.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.470+0000 7fd1fc800640 1 -- 172.21.15.82:0/2127157209 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f40ae0f0 con 0x7fd1f40ad850 2024-03-16T17:57:48.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.471+0000 7fa95d800640 1 Processor -- start 2024-03-16T17:57:48.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.471+0000 7fd1f8a00640 1 -- 172.21.15.82:0/2127157209 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722515734 0 0) 0x7fd1f40cd080 con 0x7fd1f41613b0 2024-03-16T17:57:48.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.471+0000 7fe83f400640 1 Processor -- start 2024-03-16T17:57:48.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.471+0000 7f3bcb200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3bc40cc800 0x7f3bc40ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.471+0000 7f3bcb200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3bc40b2d90 0x7f3bc40b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.471+0000 7fd1f9400640 1 --2- 172.21.15.82:0/2127157209 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1f40ad850 0x7fd1f40cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.471+0000 7fd1f8a00640 1 -- 172.21.15.82:0/2127157209 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1dc003520 con 0x7fd1f41613b0 2024-03-16T17:57:48.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.471+0000 7fd1fa800640 1 --2- 172.21.15.82:0/2127157209 >> v2:172.21.15.90:3300/0 conn(0x7fd1f40ccc50 0x7fd1f40b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.471+0000 7f3bc3e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3bc406b960 0x7f3bc41401f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56204/0 (socket says 172.21.15.82:56204) 2024-03-16T17:57:48.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.472+0000 7f3bc3e00640 1 -- 172.21.15.82:0/1411563051 learned_addr learned my addr 172.21.15.82:0/1411563051 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:48.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.472+0000 7f3bc9200640 1 --2- 172.21.15.82:0/1411563051 >> v2:172.21.15.82:3300/0 conn(0x7f3bc40b2d90 0x7f3bc40b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.472+0000 7f3bcb200640 1 -- 172.21.15.82:0/1411563051 --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc415dee0 con 0x7f3bc406b960 2024-03-16T17:57:48.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.472+0000 7f3bcb200640 1 -- 172.21.15.82:0/1411563051 --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc415e880 con 0x7f3bc40b2d90 2024-03-16T17:57:48.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.472+0000 7f3bcb200640 1 -- 172.21.15.82:0/1411563051 --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc40b4970 con 0x7f3bc40cc800 2024-03-16T17:57:48.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.472+0000 7fd1fa800640 1 -- 172.21.15.82:0/2127157209 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1f40ad850 msgr2=0x7fd1f40cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.472+0000 7fd1fa800640 1 --2- 172.21.15.82:0/2127157209 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1f40ad850 0x7fd1f40cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.472+0000 7fd1fa800640 1 -- 172.21.15.82:0/2127157209 >> v1:172.21.15.82:6789/0 conn(0x7fd1f41613b0 legacy=0x7fd1f40b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.472+0000 7fd1fa800640 1 -- 172.21.15.82:0/2127157209 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f40ae3d0 con 0x7fd1f40ccc50 2024-03-16T17:57:48.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.472+0000 7fe83f400640 1 -- start start 2024-03-16T17:57:48.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.473+0000 7f3bc3400640 1 --2- 172.21.15.82:0/1411563051 >> v2:172.21.15.90:3300/0 conn(0x7f3bc40cc800 0x7f3bc40ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.473+0000 7f3bc2a00640 1 -- 172.21.15.82:0/1411563051 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016321459 0 0) 0x7f3bc415dee0 con 0x7f3bc406b960 2024-03-16T17:57:48.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.473+0000 7f3bc2a00640 1 -- 172.21.15.82:0/1411563051 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ba8003520 con 0x7f3bc406b960 2024-03-16T17:57:48.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.471+0000 7fa95d800640 1 -- start start 2024-03-16T17:57:48.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.473+0000 7fa95d800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa9580b31c0 0x7fa958145b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.473+0000 7fa95d800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa9581455c0 0x7fa958147a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.473+0000 7fa95d800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa958148820 con 0x7fa9581451e0 2024-03-16T17:57:48.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.473+0000 7fa95d800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa958148b00 con 0x7fa9580b31c0 2024-03-16T17:57:48.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.473+0000 7fa95d800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9580cdaa0 con 0x7fa9581455c0 2024-03-16T17:57:48.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7fcfcda00640 1 Processor -- start 2024-03-16T17:57:48.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7fabcd400640 1 Processor -- start 2024-03-16T17:57:48.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7fa956000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa9580b31c0 0x7fa958145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7fa956000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa9580b31c0 0x7fa958145b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35286/0 (socket says 172.21.15.82:35286) 2024-03-16T17:57:48.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7fa956000640 1 -- 172.21.15.82:0/832693165 learned_addr learned my addr 172.21.15.82:0/832693165 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7fabcd400640 1 -- start start 2024-03-16T17:57:48.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7f3bc2a00640 1 -- 172.21.15.82:0/1411563051 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4282110969 0 0) 0x7f3ba8003520 con 0x7f3bc406b960 2024-03-16T17:57:48.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7f3bc9200640 1 -- 172.21.15.82:0/1411563051 >> v2:172.21.15.90:3300/0 conn(0x7f3bc40cc800 msgr2=0x7f3bc40ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7fa957400640 1 --2- 172.21.15.82:0/832693165 >> v2:172.21.15.90:3300/0 conn(0x7fa9581455c0 0x7fa958147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7f3bc9200640 1 --2- 172.21.15.82:0/1411563051 >> v2:172.21.15.90:3300/0 conn(0x7f3bc40cc800 0x7f3bc40ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7f3bc9200640 1 -- 172.21.15.82:0/1411563051 >> v1:172.21.15.82:6789/0 conn(0x7f3bc406b960 legacy=0x7f3bc41401f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7f3bc9200640 1 -- 172.21.15.82:0/1411563051 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc415e1c0 con 0x7f3bc40b2d90 2024-03-16T17:57:48.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.475+0000 7fa956000640 1 -- 172.21.15.82:0/832693165 >> v2:172.21.15.90:3300/0 conn(0x7fa9581455c0 msgr2=0x7fa958147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.475+0000 7fa956000640 1 --2- 172.21.15.82:0/832693165 >> v2:172.21.15.90:3300/0 conn(0x7fa9581455c0 0x7fa958147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.475+0000 7fa956000640 1 -- 172.21.15.82:0/832693165 >> v1:172.21.15.82:6789/0 conn(0x7fa9581451e0 legacy=0x7fa9580b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7f8223a00640 1 Processor -- start 2024-03-16T17:57:48.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.474+0000 7fcfcda00640 1 -- start start 2024-03-16T17:57:48.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fcfcda00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d1f60 0x7fcfc80d4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fcfcda00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fcfc80d2340 0x7fcfc80d2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fcfcda00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc80d2cc0 con 0x7fcfc815de90 2024-03-16T17:57:48.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fcfcda00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc80b0aa0 con 0x7fcfc80d2340 2024-03-16T17:57:48.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fcfcda00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc80cf4a0 con 0x7fcfc80d1f60 2024-03-16T17:57:48.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fd1fa800640 1 --2- 172.21.15.82:0/2127157209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1f40ccc50 0x7fd1f40b3390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009d90 tx=0x7fd1f002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92415b12b8816e0f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fcfc6a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d1f60 0x7fcfc80d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7f8223a00640 1 -- start start 2024-03-16T17:57:48.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fabcd400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fabc806db20 0x7fabc80b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fabcd400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc8140840 con 0x7fabc8140460 2024-03-16T17:57:48.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fabcd400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc80b39e0 con 0x7fabc80ccf10 2024-03-16T17:57:48.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fabcd400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc815e880 con 0x7fabc806db20 2024-03-16T17:57:48.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.475+0000 7fe83f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8380b2ff0 0x7fe838072830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.475+0000 7fe83f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 0x7fe838158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.475+0000 7fe83f400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8381455c0 0x7fe838149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.475+0000 7fe83f400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83814a400 con 0x7fe8381451e0 2024-03-16T17:57:48.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7f3bc9200640 1 --2- 172.21.15.82:0/1411563051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc40b2d90 0x7f3bc40b31a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40b45e0 tx=0x7f3bbc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fdd3453c7c0e4a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fcfc6a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d1f60 0x7fcfc80d4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60146/0 (socket says 172.21.15.82:60146) 2024-03-16T17:57:48.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.475+0000 7fe83f400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe838158dc0 con 0x7fe8380b2ff0 2024-03-16T17:57:48.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.475+0000 7fe83f400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe838149b90 con 0x7fe8381455c0 2024-03-16T17:57:48.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fe837e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 0x7fe838158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fe837e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 0x7fe838158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35294/0 (socket says 172.21.15.82:35294) 2024-03-16T17:57:48.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.476+0000 7fe837e00640 1 -- 172.21.15.82:0/2009080529 learned_addr learned my addr 172.21.15.82:0/2009080529 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fcfc6a00640 1 -- 172.21.15.82:0/1664107931 learned_addr learned my addr 172.21.15.82:0/1664107931 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.475+0000 7fa956000640 1 -- 172.21.15.82:0/832693165 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9580cdd80 con 0x7fa9580b31c0 2024-03-16T17:57:48.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fe83ca00640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8380b2ff0 0x7fe838072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fe837e00640 1 -- 172.21.15.82:0/2009080529 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8381455c0 msgr2=0x7fe838149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fabc6800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fabc8140460 0x7fabc80cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56214/0 (socket says 172.21.15.82:56214) 2024-03-16T17:57:48.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fabc6800640 1 -- 172.21.15.82:0/2615822472 learned_addr learned my addr 172.21.15.82:0/2615822472 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:48.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fd1f8a00640 1 -- 172.21.15.82:0/2127157209 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f002cc50 con 0x7fd1f40ccc50 2024-03-16T17:57:48.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fcfc6000640 1 --2- 172.21.15.82:0/1664107931 >> v2:172.21.15.90:3300/0 conn(0x7fcfc80d2340 0x7fcfc80d2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7f3bc2a00640 1 -- 172.21.15.82:0/1411563051 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb4001670 con 0x7f3bc40b2d90 2024-03-16T17:57:48.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fd1f8a00640 1 -- 172.21.15.82:0/2127157209 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f002cdb0 con 0x7fd1f40ccc50 2024-03-16T17:57:48.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fd1f8a00640 1 -- 172.21.15.82:0/2127157209 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0034640 con 0x7fd1f40ccc50 2024-03-16T17:57:48.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fabc7200640 1 --2- 172.21.15.82:0/2615822472 >> v2:172.21.15.90:3300/0 conn(0x7fabc806db20 0x7fabc80b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7f8223a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f821c0d28c0 0x7f821c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7f8223a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c0d4cb0 con 0x7f821c15e190 2024-03-16T17:57:48.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fabc5400640 1 -- 172.21.15.82:0/2615822472 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223794448 0 0) 0x7fabc8140840 con 0x7fabc8140460 2024-03-16T17:57:48.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fabc5400640 1 -- 172.21.15.82:0/2615822472 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabac003520 con 0x7fabc8140460 2024-03-16T17:57:48.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7f8223a00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c0d1f70 con 0x7f821c0b2e30 2024-03-16T17:57:48.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7f8223a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c0d49d0 con 0x7f821c0d28c0 2024-03-16T17:57:48.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7f3bc2a00640 1 -- 172.21.15.82:0/1411563051 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc02c950 con 0x7f3bc40b2d90 2024-03-16T17:57:48.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7f3bc2a00640 1 -- 172.21.15.82:0/1411563051 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc02cce0 con 0x7f3bc40b2d90 2024-03-16T17:57:48.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fe837e00640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8381455c0 0x7fe838149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fe837e00640 1 -- 172.21.15.82:0/2009080529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8380b2ff0 msgr2=0x7fe838072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fe837e00640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8380b2ff0 0x7fe838072830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fe837e00640 1 -- 172.21.15.82:0/2009080529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe830009b30 con 0x7fe8381451e0 2024-03-16T17:57:48.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fe837e00640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 0x7fe838158880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe82800b410 tx=0x7fe82800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.477+0000 7fcfc5600640 1 -- 172.21.15.82:0/1664107931 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44770389 0 0) 0x7fcfc80d2cc0 con 0x7fcfc815de90 2024-03-16T17:57:48.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fcfc5600640 1 -- 172.21.15.82:0/1664107931 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfac003520 con 0x7fcfc815de90 2024-03-16T17:57:48.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fabc7200640 1 -- 172.21.15.82:0/2615822472 >> v1:172.21.15.90:6789/0 conn(0x7fabc80ccf10 legacy=0x7fabc80b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fabc7200640 1 -- 172.21.15.82:0/2615822472 >> v1:172.21.15.82:6789/0 conn(0x7fabc8140460 legacy=0x7fabc80cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fabc7200640 1 -- 172.21.15.82:0/2615822472 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc815dc00 con 0x7fabc806db20 2024-03-16T17:57:48.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fe834c00640 1 -- 172.21.15.82:0/2009080529 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828011830 con 0x7fe8381451e0 2024-03-16T17:57:48.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fe834c00640 1 -- 172.21.15.82:0/2009080529 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe828011990 con 0x7fe8381451e0 2024-03-16T17:57:48.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fe834c00640 1 -- 172.21.15.82:0/2009080529 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828011d40 con 0x7fe8381451e0 2024-03-16T17:57:48.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fd1fc800640 1 -- 172.21.15.82:0/2127157209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1f40ccc50 msgr2=0x7fd1f40b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7fd1fc800640 1 --2- 172.21.15.82:0/2127157209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1f40ccc50 0x7fd1f40b3390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009d90 tx=0x7fd1f002bea0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fe83f400640 1 -- 172.21.15.82:0/2009080529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe838149e60 con 0x7fe8381451e0 2024-03-16T17:57:48.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fe83f400640 1 -- 172.21.15.82:0/2009080529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83814a8f0 con 0x7fe8381451e0 2024-03-16T17:57:48.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7f3bcb200640 1 -- 172.21.15.82:0/1411563051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc40b2d90 msgr2=0x7f3bc40b31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7f3bcb200640 1 --2- 172.21.15.82:0/1411563051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc40b2d90 0x7f3bc40b31a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3bc40b45e0 tx=0x7f3bbc02be20 comp rx=0 tx=0).stop 2024-03-16T17:57:48.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7f3bcb200640 1 -- 172.21.15.82:0/1411563051 shutdown_connections 2024-03-16T17:57:48.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7f3bcb200640 1 --2- 172.21.15.82:0/1411563051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc40b2d90 0x7f3bc40b31a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7f3bcb200640 1 --2- 172.21.15.82:0/1411563051 >> v2:172.21.15.90:3300/0 conn(0x7f3bc40cc800 0x7f3bc40ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7f3bcb200640 1 -- 172.21.15.82:0/1411563051 >> 172.21.15.82:0/1411563051 conn(0x7f3bc406ae70 msgr2=0x7f3bc406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fd1fc800640 1 -- 172.21.15.82:0/2127157209 shutdown_connections 2024-03-16T17:57:48.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fd1fc800640 1 --2- 172.21.15.82:0/2127157209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1f40ccc50 0x7fd1f40b3390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fd1fc800640 1 --2- 172.21.15.82:0/2127157209 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1f40ad850 0x7fd1f40cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fd1fc800640 1 -- 172.21.15.82:0/2127157209 >> 172.21.15.82:0/2127157209 conn(0x7fd1f406ae70 msgr2=0x7fd1f406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fd1fc800640 1 -- 172.21.15.82:0/2127157209 shutdown_connections 2024-03-16T17:57:48.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fcfc6a00640 1 -- 172.21.15.82:0/1664107931 >> v2:172.21.15.90:3300/0 conn(0x7fcfc80d2340 msgr2=0x7fcfc80d2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fcfc6a00640 1 --2- 172.21.15.82:0/1664107931 >> v2:172.21.15.90:3300/0 conn(0x7fcfc80d2340 0x7fcfc80d2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fcfc6a00640 1 -- 172.21.15.82:0/1664107931 >> v1:172.21.15.90:6789/0 conn(0x7fcfc815de90 legacy=0x7fcfc815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fcfc6a00640 1 -- 172.21.15.82:0/1664107931 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc80cf780 con 0x7fcfc80d1f60 2024-03-16T17:57:48.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fcfc6000640 1 --2- 172.21.15.82:0/1664107931 >> v2:172.21.15.90:3300/0 conn(0x7fcfc80d2340 0x7fcfc80d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:48.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.478+0000 7f8221000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f821c15e190 0x7f821c0d21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56230/0 (socket says 172.21.15.82:56230) 2024-03-16T17:57:48.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7f8221000640 1 -- 172.21.15.82:0/333903286 learned_addr learned my addr 172.21.15.82:0/333903286 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:48.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7f821be00640 1 --2- 172.21.15.82:0/333903286 >> v2:172.21.15.90:3300/0 conn(0x7f821c0d28c0 0x7f821c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fa956000640 1 --2- 172.21.15.82:0/832693165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9580b31c0 0x7fa958145b10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9580af550 tx=0x7fa94802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcccc69ddcead18 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7f3bcb200640 1 -- 172.21.15.82:0/1411563051 shutdown_connections 2024-03-16T17:57:48.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.480+0000 7f3bcb200640 1 -- 172.21.15.82:0/1411563051 wait complete. 2024-03-16T17:57:48.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.480+0000 7fabc7200640 1 --2- 172.21.15.82:0/2615822472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabc806db20 0x7fabc80b34a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009d90 tx=0x7fabc002e150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a9b8cdc6a0c3c3e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.480+0000 7fa955600640 1 -- 172.21.15.82:0/832693165 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948030860 con 0x7fa9580b31c0 2024-03-16T17:57:48.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7fa955600640 1 -- 172.21.15.82:0/832693165 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9480309c0 con 0x7fa9580b31c0 2024-03-16T17:57:48.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7fa955600640 1 -- 172.21.15.82:0/832693165 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa948030d50 con 0x7fa9580b31c0 2024-03-16T17:57:48.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.480+0000 7fcfc6a00640 1 --2- 172.21.15.82:0/1664107931 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d1f60 0x7fcfc80d4910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009e50 tx=0x7fcfc002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=808e39e8f0c1061d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7fabc5400640 1 -- 172.21.15.82:0/2615822472 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0036440 con 0x7fabc806db20 2024-03-16T17:57:48.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7fabc5400640 1 -- 172.21.15.82:0/2615822472 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc00365a0 con 0x7fabc806db20 2024-03-16T17:57:48.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7fabc5400640 1 -- 172.21.15.82:0/2615822472 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc0036930 con 0x7fabc806db20 2024-03-16T17:57:48.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7fcfc5600640 1 -- 172.21.15.82:0/1664107931 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc002ca80 con 0x7fcfc80d1f60 2024-03-16T17:57:48.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7fcfc5600640 1 -- 172.21.15.82:0/1664107931 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc002cbe0 con 0x7fcfc80d1f60 2024-03-16T17:57:48.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7fcfc5600640 1 -- 172.21.15.82:0/1664107931 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00343f0 con 0x7fcfc80d1f60 2024-03-16T17:57:48.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fe82e000640 1 -- 172.21.15.82:0/2009080529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80c005290 con 0x7fe8381451e0 2024-03-16T17:57:48.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fa95d800640 1 -- 172.21.15.82:0/832693165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9580b31c0 msgr2=0x7fa958145b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fa95d800640 1 --2- 172.21.15.82:0/832693165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9580b31c0 0x7fa958145b10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa9580af550 tx=0x7fa94802b9a0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fabcd400640 1 -- 172.21.15.82:0/2615822472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabc806db20 msgr2=0x7fabc80b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fabcd400640 1 --2- 172.21.15.82:0/2615822472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabc806db20 0x7fabc80b34a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009d90 tx=0x7fabc002e150 comp rx=0 tx=0).stop 2024-03-16T17:57:48.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.480+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706268597 0 0) 0x7f821c0d4cb0 con 0x7f821c15e190 2024-03-16T17:57:48.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.480+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8200003520 con 0x7f821c15e190 2024-03-16T17:57:48.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2640321981 0 0) 0x7f8200003520 con 0x7f821c15e190 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 >> v2:172.21.15.90:3300/0 conn(0x7f821c0d28c0 msgr2=0x7f821c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 --2- 172.21.15.82:0/333903286 >> v2:172.21.15.90:3300/0 conn(0x7f821c0d28c0 0x7f821c0ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 >> v1:172.21.15.90:6789/0 conn(0x7f821c0b2e30 legacy=0x7f821c0ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0d1c90 con 0x7f821c15e190 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f820c001690 con 0x7f821c15e190 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 >> v1:172.21.15.82:6789/0 conn(0x7f821c15e190 legacy=0x7f821c0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8200003970 0x7f8200003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82000047c0 0x7f8200004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8200005380 0x7f8200014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0d49d0 con 0x7f8200003970 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8200005bb0 con 0x7f82000047c0 2024-03-16T17:57:48.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.481+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8200005d70 con 0x7f8200005380 2024-03-16T17:57:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fa95d800640 1 -- 172.21.15.82:0/832693165 shutdown_connections 2024-03-16T17:57:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fa95d800640 1 --2- 172.21.15.82:0/832693165 >> v2:172.21.15.90:3300/0 conn(0x7fa9581455c0 0x7fa958147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fa95d800640 1 --2- 172.21.15.82:0/832693165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9580b31c0 0x7fa958145b10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fa95d800640 1 -- 172.21.15.82:0/832693165 >> 172.21.15.82:0/832693165 conn(0x7fa9580b17b0 msgr2=0x7fa9580b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fabcd400640 1 -- 172.21.15.82:0/2615822472 shutdown_connections 2024-03-16T17:57:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fabcd400640 1 --2- 172.21.15.82:0/2615822472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabc806db20 0x7fabc80b34a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fabcd400640 1 -- 172.21.15.82:0/2615822472 >> 172.21.15.82:0/2615822472 conn(0x7fabc806ae70 msgr2=0x7fabc806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fabcd400640 1 -- 172.21.15.82:0/2615822472 shutdown_connections 2024-03-16T17:57:48.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fcfcda00640 1 -- 172.21.15.82:0/1664107931 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d1f60 msgr2=0x7fcfc80d4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fcfcda00640 1 --2- 172.21.15.82:0/1664107931 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d1f60 0x7fcfc80d4910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009e50 tx=0x7fcfc002bee0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7f8221a00640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8200005380 0x7f8200014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7f8221000640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8200003970 0x7f8200003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7fcfcda00640 1 -- 172.21.15.82:0/1664107931 shutdown_connections 2024-03-16T17:57:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7fcfcda00640 1 --2- 172.21.15.82:0/1664107931 >> v2:172.21.15.90:3300/0 conn(0x7fcfc80d2340 0x7fcfc80d2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7fcfcda00640 1 --2- 172.21.15.82:0/1664107931 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d1f60 0x7fcfc80d4910 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7fcfcda00640 1 -- 172.21.15.82:0/1664107931 >> 172.21.15.82:0/1664107931 conn(0x7fcfc80d3df0 msgr2=0x7fcfc80d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.479+0000 7fd1fc800640 1 -- 172.21.15.82:0/2127157209 wait complete. 2024-03-16T17:57:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fd1fc800640 1 Processor -- start 2024-03-16T17:57:48.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.482+0000 7fd1fc800640 1 -- start start 2024-03-16T17:57:48.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7f821be00640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82000047c0 0x7f8200004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7fa95d800640 1 -- 172.21.15.82:0/832693165 shutdown_connections 2024-03-16T17:57:48.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7fa95d800640 1 -- 172.21.15.82:0/832693165 wait complete. 2024-03-16T17:57:48.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7fe834c00640 1 -- 172.21.15.82:0/2009080529 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe82801a400 con 0x7fe8381451e0 2024-03-16T17:57:48.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7f3bcb200640 1 Processor -- start 2024-03-16T17:57:48.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.484+0000 7fcfcda00640 1 -- 172.21.15.82:0/1664107931 shutdown_connections 2024-03-16T17:57:48.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.484+0000 7fcfcda00640 1 -- 172.21.15.82:0/1664107931 wait complete. 2024-03-16T17:57:48.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7f8221000640 1 -- 172.21.15.82:0/333903286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8200005380 msgr2=0x7f8200014190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7fabcd400640 1 -- 172.21.15.82:0/2615822472 wait complete. 2024-03-16T17:57:48.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.484+0000 7f3bcb200640 1 -- start start 2024-03-16T17:57:48.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fe834c00640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe81806b480 0x7fe81806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7f8221000640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8200005380 0x7f8200014190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7f8221000640 1 -- 172.21.15.82:0/333903286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82000047c0 msgr2=0x7f8200004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7f8221000640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82000047c0 0x7f8200004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.483+0000 7f8221000640 1 -- 172.21.15.82:0/333903286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0d1c90 con 0x7f8200003970 2024-03-16T17:57:48.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fe834c00640 1 -- 172.21.15.82:0/2009080529 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe828089d80 con 0x7fe8381451e0 2024-03-16T17:57:48.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7f821be00640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f82000047c0 0x7f8200004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:48.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7f8221a00640 1 -- 172.21.15.82:0/333903286 reap_dead start 2024-03-16T17:57:48.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fd1fc800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f40ad850 0x7fd1f4158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fd1fc800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1f40ccc50 0x7fd1f414e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fd1fc800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1f41613b0 0x7fd1f415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fd1fc800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f414e860 con 0x7fd1f40ad850 2024-03-16T17:57:48.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fd1fc800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f40b07c0 con 0x7fd1f41613b0 2024-03-16T17:57:48.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fd1fc800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f40b0200 con 0x7fd1f40ccc50 2024-03-16T17:57:48.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fd1f9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f40ad850 0x7fd1f4158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7f3bcb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc406b960 0x7f3bc40b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fd1f9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f40ad850 0x7fd1f4158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35308/0 (socket says 172.21.15.82:35308) 2024-03-16T17:57:48.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7fd1f9e00640 1 -- 172.21.15.82:0/3671938089 learned_addr learned my addr 172.21.15.82:0/3671938089 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7f3bcb200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bc40b2d90 0x7f3bc414ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7f3bcb200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bc40cc800 0x7f3bc41504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7f3bcb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc41554a0 con 0x7f3bc406b960 2024-03-16T17:57:48.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7fd1fa800640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1f41613b0 0x7fd1f415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7f3bcb200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4155780 con 0x7f3bc40cc800 2024-03-16T17:57:48.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.485+0000 7f3bcb200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc4155a60 con 0x7f3bc40b2d90 2024-03-16T17:57:48.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7f3bc3400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bc40b2d90 0x7f3bc414ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7f3bc3400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bc40b2d90 0x7f3bc414ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60184/0 (socket says 172.21.15.82:60184) 2024-03-16T17:57:48.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7f3bc3400640 1 -- 172.21.15.82:0/1417815671 learned_addr learned my addr 172.21.15.82:0/1417815671 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7fd1f9e00640 1 -- 172.21.15.82:0/3671938089 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1f40ccc50 msgr2=0x7fd1f414e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7fd1f9e00640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1f40ccc50 0x7fd1f414e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7fd1f9e00640 1 -- 172.21.15.82:0/3671938089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1f41613b0 msgr2=0x7fd1f415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7fd1f9e00640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1f41613b0 0x7fd1f415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7fd1f9e00640 1 -- 172.21.15.82:0/3671938089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f0009a70 con 0x7fd1f40ad850 2024-03-16T17:57:48.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7fd1fa800640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1f41613b0 0x7fd1f415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:48.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.486+0000 7fd1f9e00640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f40ad850 0x7fd1f4158860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40010e0 tx=0x7fd1e400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f3bc9200640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bc40cc800 0x7f3bc41504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7fa95d800640 1 Processor -- start 2024-03-16T17:57:48.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f3bc3400640 1 -- 172.21.15.82:0/1417815671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bc40cc800 msgr2=0x7f3bc41504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f3bc3400640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bc40cc800 0x7f3bc41504a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f3bc3400640 1 -- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc406b960 msgr2=0x7f3bc40b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f3bc3e00640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc406b960 0x7f3bc40b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f3bc3400640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc406b960 0x7f3bc40b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f109ba00640 1 Processor -- start 2024-03-16T17:57:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f3bc3400640 1 -- 172.21.15.82:0/1417815671 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc009a70 con 0x7f3bc40b2d90 2024-03-16T17:57:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f3bc3e00640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc406b960 0x7f3bc40b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:48.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7fa95d800640 1 -- start start 2024-03-16T17:57:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7fabcd400640 1 Processor -- start 2024-03-16T17:57:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f8221000640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8200003970 0x7f8200003d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f820c00c260 tx=0x7f82000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6d9dca5b7139c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.487+0000 7f3bc3400640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bc40b2d90 0x7f3bc414ff60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000d5f0 tx=0x7f3bb000dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.488+0000 7fcfcda00640 1 Processor -- start 2024-03-16T17:57:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.488+0000 7f109ba00640 1 -- start start 2024-03-16T17:57:48.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.488+0000 7f3bbbe00640 1 -- 172.21.15.82:0/1417815671 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00125f0 con 0x7f3bc40b2d90 2024-03-16T17:57:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.488+0000 7f3bbbe00640 1 -- 172.21.15.82:0/1417815671 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb0012750 con 0x7f3bc40b2d90 2024-03-16T17:57:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.488+0000 7f3bbbe00640 1 -- 172.21.15.82:0/1417815671 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0012b00 con 0x7f3bc40b2d90 2024-03-16T17:57:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.488+0000 7f3bcb200640 1 -- 172.21.15.82:0/1417815671 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc4155d40 con 0x7f3bc40b2d90 2024-03-16T17:57:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.488+0000 7f3bcb200640 1 -- 172.21.15.82:0/1417815671 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc40b2400 con 0x7f3bc40b2d90 2024-03-16T17:57:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.488+0000 7fcfcda00640 1 -- start start 2024-03-16T17:57:48.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fe83ca00640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe81806b480 0x7fe81806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fa95d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9580b31c0 0x7fa958138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.488+0000 7fabcd400640 1 -- start start 2024-03-16T17:57:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fabcd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabc806db20 0x7fabc81557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fabcd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabc80ccf10 0x7fabc8153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fabcd400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabc8140460 0x7fabc8154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fabcd400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc8155cf0 con 0x7fabc80ccf10 2024-03-16T17:57:48.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fabcd400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc81ef9f0 con 0x7fabc806db20 2024-03-16T17:57:48.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fabcd400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80cbe70 con 0x7fabc8140460 2024-03-16T17:57:48.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fa95d800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa9581451e0 0x7fa958071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fa95d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9581455c0 0x7fa958072470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fa95d800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958164e20 con 0x7fa9581455c0 2024-03-16T17:57:48.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7f109ba00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109415bd10 0x7f10940cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fabc5e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabc80ccf10 0x7fabc8153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7f109ba00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f10940aed50 0x7f10940ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7f109ba00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1094162a70 con 0x7f10940b44f0 2024-03-16T17:57:48.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7f109ba00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1094161ad0 con 0x7f10940aed50 2024-03-16T17:57:48.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7f109ba00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10940cddd0 con 0x7f109415bd10 2024-03-16T17:57:48.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7f3bcb200640 1 -- 172.21.15.82:0/1417815671 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc4073420 con 0x7f3bc40b2d90 2024-03-16T17:57:48.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fa95d800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958135850 con 0x7fa9580b31c0 2024-03-16T17:57:48.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fa95d800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa958134cd0 con 0x7fa9581451e0 2024-03-16T17:57:48.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fa957400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9581455c0 0x7fa958072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fa957400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9581455c0 0x7fa958072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35332/0 (socket says 172.21.15.82:35332) 2024-03-16T17:57:48.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.489+0000 7fa957400640 1 -- 172.21.15.82:0/3689841466 learned_addr learned my addr 172.21.15.82:0/3689841466 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fa957400640 1 -- 172.21.15.82:0/3689841466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa9581451e0 msgr2=0x7fa958071f30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T17:57:48.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fabc5e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabc80ccf10 0x7fabc8153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35336/0 (socket says 172.21.15.82:35336) 2024-03-16T17:57:48.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fabc5e00640 1 -- 172.21.15.82:0/3063360628 learned_addr learned my addr 172.21.15.82:0/3063360628 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fabc6800640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabc806db20 0x7fabc81557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fa957400640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa9581451e0 0x7fa958071f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fa957400640 1 -- 172.21.15.82:0/3689841466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9580b31c0 msgr2=0x7fa958138040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T17:57:48.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fa957400640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9580b31c0 0x7fa958138040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fa957400640 1 -- 172.21.15.82:0/3689841466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9480094d0 con 0x7fa9581455c0 2024-03-16T17:57:48.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7f1099000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109415bd10 0x7f10940cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7f1099000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109415bd10 0x7f10940cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60206/0 (socket says 172.21.15.82:60206) 2024-03-16T17:57:48.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fa957400640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9581455c0 0x7fa958072470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa950007e60 tx=0x7fa95000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7f1099000640 1 -- 172.21.15.82:0/2209428595 learned_addr learned my addr 172.21.15.82:0/2209428595 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7f1093e00640 1 --2- 172.21.15.82:0/2209428595 >> v2:172.21.15.82:3300/0 conn(0x7f10940aed50 0x7f10940ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fcfcda00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc80d1f60 0x7fcfc8161080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fcfcda00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d2340 0x7fcfc80d0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fcfcda00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc815de90 0x7fcfc80d1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fcfcda00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc80cdd00 con 0x7fcfc80d1f60 2024-03-16T17:57:48.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fcfcda00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc81615c0 con 0x7fcfc815de90 2024-03-16T17:57:48.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fcfcda00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc80b74e0 con 0x7fcfc80d2340 2024-03-16T17:57:48.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.490+0000 7fabc7200640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabc8140460 0x7fabc8154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fabc6800640 1 -- 172.21.15.82:0/3063360628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabc8140460 msgr2=0x7fabc8154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fabc6800640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabc8140460 0x7fabc8154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fabc6800640 1 -- 172.21.15.82:0/3063360628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabc80ccf10 msgr2=0x7fabc8153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fabc6800640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabc80ccf10 0x7fabc8153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fabc6800640 1 -- 172.21.15.82:0/3063360628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc0009a70 con 0x7fabc806db20 2024-03-16T17:57:48.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fe83ca00640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe81806b480 0x7fe81806d940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe830035000 tx=0x7fe830030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fcfc6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc80d1f60 0x7fcfc8161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fabc6800640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabc806db20 0x7fabc81557b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fabb400d5a0 tx=0x7fabb4001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7f1093400640 1 -- 172.21.15.82:0/2209428595 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333453902 0 0) 0x7f1094162a70 con 0x7f10940b44f0 2024-03-16T17:57:48.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7f1093400640 1 -- 172.21.15.82:0/2209428595 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1078003520 con 0x7f10940b44f0 2024-03-16T17:57:48.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7f1093e00640 1 -- 172.21.15.82:0/2209428595 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109415bd10 msgr2=0x7f10940cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:48.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7f1093e00640 1 --2- 172.21.15.82:0/2209428595 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109415bd10 0x7f10940cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7f1093e00640 1 -- 172.21.15.82:0/2209428595 >> v1:172.21.15.90:6789/0 conn(0x7f10940b44f0 legacy=0x7f10940ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7f1093e00640 1 -- 172.21.15.82:0/2209428595 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109415f4f0 con 0x7f10940aed50 2024-03-16T17:57:48.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fcfc6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc80d1f60 0x7fcfc8161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35354/0 (socket says 172.21.15.82:35354) 2024-03-16T17:57:48.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.491+0000 7fcfc6a00640 1 -- 172.21.15.82:0/3992933351 learned_addr learned my addr 172.21.15.82:0/3992933351 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7f3bbbe00640 1 -- 172.21.15.82:0/1417815671 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f3bb0012c60 con 0x7f3bc40b2d90 2024-03-16T17:57:48.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fcfc7400640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc815de90 0x7fcfc80d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fcfc6000640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d2340 0x7fcfc80d0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fcfc6a00640 1 -- 172.21.15.82:0/3992933351 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d2340 msgr2=0x7fcfc80d0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fcfc6a00640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d2340 0x7fcfc80d0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fcfc6a00640 1 -- 172.21.15.82:0/3992933351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc815de90 msgr2=0x7fcfc80d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fcfc6a00640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc815de90 0x7fcfc80d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fcfc6a00640 1 -- 172.21.15.82:0/3992933351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb8009400 con 0x7fcfc80d1f60 2024-03-16T17:57:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fcfc7400640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc815de90 0x7fcfc80d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fabbea00640 1 -- 172.21.15.82:0/3063360628 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb400f630 con 0x7fabc806db20 2024-03-16T17:57:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fabbea00640 1 -- 172.21.15.82:0/3063360628 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb400f790 con 0x7fabc806db20 2024-03-16T17:57:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fabbea00640 1 -- 172.21.15.82:0/3063360628 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb400fb40 con 0x7fabc806db20 2024-03-16T17:57:48.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fabcd400640 1 -- 172.21.15.82:0/3063360628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc8156850 con 0x7fabc806db20 2024-03-16T17:57:48.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.492+0000 7fabcd400640 1 -- 172.21.15.82:0/3063360628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc8156dd0 con 0x7fabc806db20 2024-03-16T17:57:48.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.493+0000 7f1093e00640 1 --2- 172.21.15.82:0/2209428595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940aed50 0x7f10940ccc50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f108c009d90 tx=0x7f108c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67387a7a3d018763 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.493+0000 7f3bbbe00640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3ba406b3b0 0x7f3ba406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.493+0000 7f3bbbe00640 1 -- 172.21.15.82:0/1417815671 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bb008a490 con 0x7f3bc40b2d90 2024-03-16T17:57:48.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.493+0000 7fcfc6a00640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc80d1f60 0x7fcfc8161080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009e50 tx=0x7fcfc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.493+0000 7f3bc3e00640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3ba406b3b0 0x7f3ba406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.494+0000 7fabcd400640 1 -- 172.21.15.82:0/3063360628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc81ef7c0 con 0x7fabc806db20 2024-03-16T17:57:48.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.494+0000 7fd1ee000640 1 -- 172.21.15.82:0/3671938089 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4005ce0 con 0x7fd1f40ad850 2024-03-16T17:57:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.494+0000 7fd1ee000640 1 -- 172.21.15.82:0/3671938089 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e4005e40 con 0x7fd1f40ad850 2024-03-16T17:57:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.494+0000 7fd1ee000640 1 -- 172.21.15.82:0/3671938089 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40072b0 con 0x7fd1f40ad850 2024-03-16T17:57:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.494+0000 7f1093400640 1 -- 172.21.15.82:0/2209428595 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c02cbd0 con 0x7f10940aed50 2024-03-16T17:57:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.494+0000 7f1093400640 1 -- 172.21.15.82:0/2209428595 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108c02cd30 con 0x7f10940aed50 2024-03-16T17:57:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.494+0000 7f1093400640 1 -- 172.21.15.82:0/2209428595 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108c02cea0 con 0x7f10940aed50 2024-03-16T17:57:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fa94ea00640 1 -- 172.21.15.82:0/3689841466 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950014420 con 0x7fa9581455c0 2024-03-16T17:57:48.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c005ad0 con 0x7f8200003970 2024-03-16T17:57:48.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c005c30 con 0x7f8200003970 2024-03-16T17:57:48.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7f821b400640 1 -- 172.21.15.82:0/333903286 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c02cb10 con 0x7f8200003970 2024-03-16T17:57:48.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fa94ea00640 1 -- 172.21.15.82:0/3689841466 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa950014580 con 0x7fa9581455c0 2024-03-16T17:57:48.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fe834c00640 1 -- 172.21.15.82:0/2009080529 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe8280549c0 con 0x7fe8381451e0 2024-03-16T17:57:48.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fcfbea00640 1 -- 172.21.15.82:0/3992933351 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc003e450 con 0x7fcfc80d1f60 2024-03-16T17:57:48.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7f3bc3e00640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3ba406b3b0 0x7f3ba406d870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4001380 tx=0x7f3bb4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7f109ba00640 1 -- 172.21.15.82:0/2209428595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940aed50 msgr2=0x7f10940ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7f109ba00640 1 --2- 172.21.15.82:0/2209428595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940aed50 0x7f10940ccc50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f108c009d90 tx=0x7f108c02be20 comp rx=0 tx=0).stop 2024-03-16T17:57:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fcfbea00640 1 -- 172.21.15.82:0/3992933351 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc003e5d0 con 0x7fcfc80d1f60 2024-03-16T17:57:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fcfbea00640 1 -- 172.21.15.82:0/3992933351 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc003e980 con 0x7fcfc80d1f60 2024-03-16T17:57:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7f109ba00640 1 -- 172.21.15.82:0/2209428595 shutdown_connections 2024-03-16T17:57:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7f109ba00640 1 --2- 172.21.15.82:0/2209428595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940aed50 0x7f10940ccc50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7f109ba00640 1 --2- 172.21.15.82:0/2209428595 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f109415bd10 0x7f10940cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7f109ba00640 1 -- 172.21.15.82:0/2209428595 >> 172.21.15.82:0/2209428595 conn(0x7f109406ae70 msgr2=0x7f109406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.496+0000 7f109ba00640 1 -- 172.21.15.82:0/2209428595 shutdown_connections 2024-03-16T17:57:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fa94ea00640 1 -- 172.21.15.82:0/3689841466 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9500148f0 con 0x7fa9581455c0 2024-03-16T17:57:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fa95d800640 1 -- 172.21.15.82:0/3689841466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa958134fb0 con 0x7fa9581455c0 2024-03-16T17:57:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fa95d800640 1 -- 172.21.15.82:0/3689841466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa958135570 con 0x7fa9581455c0 2024-03-16T17:57:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.496+0000 7fcfcda00640 1 -- 172.21.15.82:0/3992933351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc0009b30 con 0x7fcfc80d1f60 2024-03-16T17:57:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fd1fc800640 1 -- 172.21.15.82:0/3671938089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f40b04e0 con 0x7fd1f40ad850 2024-03-16T17:57:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.495+0000 7fd1fc800640 1 -- 172.21.15.82:0/3671938089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f4158e40 con 0x7fd1f40ad850 2024-03-16T17:57:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.497+0000 7fcfcda00640 1 -- 172.21.15.82:0/3992933351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc80b6e90 con 0x7fcfc80d1f60 2024-03-16T17:57:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.496+0000 7f109ba00640 1 -- 172.21.15.82:0/2209428595 wait complete. 2024-03-16T17:57:48.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.497+0000 7f109ba00640 1 Processor -- start 2024-03-16T17:57:48.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.498+0000 7f8223a00640 1 -- 172.21.15.82:0/333903286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8200003970 msgr2=0x7f8200003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.498+0000 7f8223a00640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8200003970 0x7f8200003d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f820c00c260 tx=0x7f82000057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.498+0000 7fa95d800640 1 -- 172.21.15.82:0/3689841466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa958073420 con 0x7fa9581455c0 2024-03-16T17:57:48.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.498+0000 7fd1fc800640 1 -- 172.21.15.82:0/3671938089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f40b0dd0 con 0x7fd1f40ad850 2024-03-16T17:57:48.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.498+0000 7fcfcda00640 1 -- 172.21.15.82:0/3992933351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc80a1e20 con 0x7fcfc80d1f60 2024-03-16T17:57:48.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.498+0000 7fabbea00640 1 -- 172.21.15.82:0/3063360628 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fabb400fca0 con 0x7fabc806db20 2024-03-16T17:57:48.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.498+0000 7f109ba00640 1 -- start start 2024-03-16T17:57:48.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.499+0000 7f109ba00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f10940aed50 0x7f1094158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.499+0000 7f109ba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940b44f0 0x7f10941441e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.499+0000 7f109ba00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1094146240 0x7f10941466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.499+0000 7f1093e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940b44f0 0x7f10941441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.500+0000 7f1093e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940b44f0 0x7f10941441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35362/0 (socket says 172.21.15.82:35362) 2024-03-16T17:57:48.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.500+0000 7fabbea00640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faba806b3b0 0x7faba806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.500+0000 7f1093e00640 1 -- 172.21.15.82:0/3990588651 learned_addr learned my addr 172.21.15.82:0/3990588651 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.500+0000 7f1099000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f10940aed50 0x7f1094158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.500+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10940b26a0 con 0x7f10940b44f0 2024-03-16T17:57:48.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.500+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1094158c70 con 0x7f1094146240 2024-03-16T17:57:48.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.500+0000 7f1099a00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1094146240 0x7f10941466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.500+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1094146c30 con 0x7f10940aed50 2024-03-16T17:57:48.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7f8223a00640 1 -- 172.21.15.82:0/333903286 shutdown_connections 2024-03-16T17:57:48.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7f8223a00640 1 --2- 172.21.15.82:0/333903286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8200003970 0x7f8200003d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7f8223a00640 1 -- 172.21.15.82:0/333903286 >> 172.21.15.82:0/333903286 conn(0x7f821c0d3d80 msgr2=0x7f821c0d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7f1099a00640 1 -- 172.21.15.82:0/3990588651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f10940aed50 msgr2=0x7f1094158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7f1099a00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f10940aed50 0x7f1094158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7f1099a00640 1 -- 172.21.15.82:0/3990588651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940b44f0 msgr2=0x7f10941441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7f1099a00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940b44f0 0x7f10941441e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7f1099a00640 1 -- 172.21.15.82:0/3990588651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c009a70 con 0x7f1094146240 2024-03-16T17:57:48.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7f1093e00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940b44f0 0x7f10941441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:48.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7fd1ee000640 1 -- 172.21.15.82:0/3671938089 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fd1e40074d0 con 0x7fd1f40ad850 2024-03-16T17:57:48.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7fa94ea00640 1 -- 172.21.15.82:0/3689841466 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa950014a50 con 0x7fa9581455c0 2024-03-16T17:57:48.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.502+0000 7fa94ea00640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa93406b480 0x7fa93406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.500+0000 7fabbea00640 1 -- 172.21.15.82:0/3063360628 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabb408a2f0 con 0x7fabc806db20 2024-03-16T17:57:48.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.502+0000 7fabc5e00640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faba806b3b0 0x7faba806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.501+0000 7f1099a00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1094146240 0x7f10941466f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f108400c480 tx=0x7f108400c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.502+0000 7fd1ee000640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd1d806b3b0 0x7fd1d806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.503+0000 7fa94ea00640 1 -- 172.21.15.82:0/3689841466 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa95008a450 con 0x7fa9581455c0 2024-03-16T17:57:48.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.502+0000 7fd1ee000640 1 -- 172.21.15.82:0/3671938089 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1e4089340 con 0x7fd1f40ad850 2024-03-16T17:57:48.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.503+0000 7fa956a00640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa93406b480 0x7fa93406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.503+0000 7fd1f9400640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd1d806b3b0 0x7fd1d806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.503+0000 7fabc5e00640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faba806b3b0 0x7faba806d870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabc8153d30 tx=0x7fabb8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.504+0000 7f8223a00640 1 -- 172.21.15.82:0/333903286 shutdown_connections 2024-03-16T17:57:48.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.505+0000 7fa956a00640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa93406b480 0x7fa93406d940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa940000f60 tx=0x7fa9400079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.504+0000 7f8223a00640 1 -- 172.21.15.82:0/333903286 wait complete. 2024-03-16T17:57:48.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.502+0000 7fcfbea00640 1 -- 172.21.15.82:0/3992933351 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fcfc002cd50 con 0x7fcfc80d1f60 2024-03-16T17:57:48.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.505+0000 7f3bbbe00640 1 -- 172.21.15.82:0/1417815671 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3bb00551d0 con 0x7f3bc40b2d90 2024-03-16T17:57:48.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.505+0000 7fd1f9400640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd1d806b3b0 0x7fd1d806d870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8006fd0 tx=0x7fd1e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.506+0000 7fcfbea00640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfa806b3e0 0x7fcfa806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.506+0000 7fcfbea00640 1 -- 172.21.15.82:0/3992933351 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfc00ad0e0 con 0x7fcfc80d1f60 2024-03-16T17:57:48.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.506+0000 7fcfc6000640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfa806b3e0 0x7fcfa806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.506+0000 7f8223a00640 1 Processor -- start 2024-03-16T17:57:48.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.507+0000 7f8223a00640 1 -- start start 2024-03-16T17:57:48.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.507+0000 7fcfc6000640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfa806b3e0 0x7fcfa806d8a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcfc81fb360 tx=0x7fcfb8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.508+0000 7f8223a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8200003970 0x7f821c0b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.508+0000 7f8223a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f821c14de20 0x7f821c14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.508+0000 7f8223a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f821c156180 0x7f821c156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.508+0000 7fd1ee000640 1 -- 172.21.15.82:0/3671938089 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd1e4054280 con 0x7fd1f40ad850 2024-03-16T17:57:48.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.508+0000 7f8223a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c14e7e0 con 0x7f821c156180 2024-03-16T17:57:48.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.508+0000 7f8223a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c156eb0 con 0x7f821c14de20 2024-03-16T17:57:48.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.508+0000 7f8223a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c15ea40 con 0x7f8200003970 2024-03-16T17:57:48.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.509+0000 7f8221000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8200003970 0x7f821c0b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.509+0000 7f8221a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f821c156180 0x7f821c156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.509+0000 7f821be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f821c14de20 0x7f821c14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.509+0000 7f8221000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8200003970 0x7f821c0b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60216/0 (socket says 172.21.15.82:60216) 2024-03-16T17:57:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.509+0000 7f8221000640 1 -- 172.21.15.82:0/3100874279 learned_addr learned my addr 172.21.15.82:0/3100874279 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.510+0000 7f821be00640 1 -- 172.21.15.82:0/3100874279 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8200003970 msgr2=0x7f821c0b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.510+0000 7f821be00640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8200003970 0x7f821c0b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.510+0000 7f821be00640 1 -- 172.21.15.82:0/3100874279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f821c156180 msgr2=0x7f821c156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.510+0000 7f821be00640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f821c156180 0x7f821c156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.510+0000 7f821be00640 1 -- 172.21.15.82:0/3100874279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c15ed20 con 0x7f821c14de20 2024-03-16T17:57:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.510+0000 7f821be00640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f821c14de20 0x7f821c14e2a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f820800f4e0 tx=0x7f820800f9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.513+0000 7f8218c00640 1 -- 172.21.15.82:0/3100874279 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208019070 con 0x7f821c14de20 2024-03-16T17:57:48.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.513+0000 7f8223a00640 1 -- 172.21.15.82:0/3100874279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c15f000 con 0x7f821c14de20 2024-03-16T17:57:48.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.513+0000 7f1090c00640 1 -- 172.21.15.82:0/3990588651 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084005480 con 0x7f1094146240 2024-03-16T17:57:48.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.513+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1094200c40 con 0x7f1094146240 2024-03-16T17:57:48.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.513+0000 7f8223a00640 1 -- 172.21.15.82:0/3100874279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c0c7a80 con 0x7f821c14de20 2024-03-16T17:57:48.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.513+0000 7f8218c00640 1 -- 172.21.15.82:0/3100874279 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8208004240 con 0x7f821c14de20 2024-03-16T17:57:48.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.513+0000 7f8218c00640 1 -- 172.21.15.82:0/3100874279 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208003610 con 0x7f821c14de20 2024-03-16T17:57:48.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.513+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1094159780 con 0x7f1094146240 2024-03-16T17:57:48.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.514+0000 7fabbea00640 1 -- 172.21.15.82:0/3063360628 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fabb4054fe0 con 0x7fabc806db20 2024-03-16T17:57:48.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.514+0000 7f1090c00640 1 -- 172.21.15.82:0/3990588651 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10840035f0 con 0x7f1094146240 2024-03-16T17:57:48.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.514+0000 7f1090c00640 1 -- 172.21.15.82:0/3990588651 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10840039a0 con 0x7f1094146240 2024-03-16T17:57:48.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.515+0000 7f8223a00640 1 -- 172.21.15.82:0/3100874279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f4005290 con 0x7f821c14de20 2024-03-16T17:57:48.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.516+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1094073420 con 0x7f1094146240 2024-03-16T17:57:48.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.517+0000 7f8218c00640 1 -- 172.21.15.82:0/3100874279 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8208005990 con 0x7f821c14de20 2024-03-16T17:57:48.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.518+0000 7f8218c00640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81fc06b3e0 0x7f81fc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.518+0000 7f1090c00640 1 -- 172.21.15.82:0/3990588651 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f1084003b00 con 0x7f1094146240 2024-03-16T17:57:48.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.520+0000 7f1090c00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f107406b3b0 0x7f107406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:48.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.520+0000 7f8218c00640 1 -- 172.21.15.82:0/3100874279 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8208089a90 con 0x7f821c14de20 2024-03-16T17:57:48.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.520+0000 7f1090c00640 1 -- 172.21.15.82:0/3990588651 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1084089260 con 0x7f1094146240 2024-03-16T17:57:48.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.520+0000 7f8221000640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81fc06b3e0 0x7f81fc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.520+0000 7f1099000640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f107406b3b0 0x7f107406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:48.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.521+0000 7fcfbea00640 1 -- 172.21.15.82:0/3992933351 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcfc0077f90 con 0x7fcfc80d1f60 2024-03-16T17:57:48.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.522+0000 7f8221000640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81fc06b3e0 0x7f81fc06d8a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f820c000c00 tx=0x7f820c02f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.522+0000 7f1099000640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f107406b3b0 0x7f107406d870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1080009b30 tx=0x7f1080009240 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:48.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.529+0000 7fa94ea00640 1 -- 172.21.15.82:0/3689841466 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa950055120 con 0x7fa9581455c0 2024-03-16T17:57:48.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.533+0000 7f8218c00640 1 -- 172.21.15.82:0/3100874279 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8208054780 con 0x7f821c14de20 2024-03-16T17:57:48.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.550+0000 7f1090c00640 1 -- 172.21.15.82:0/3990588651 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f10840540e0 con 0x7f1094146240 2024-03-16T17:57:48.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.890+0000 7fe82e000640 1 -- 172.21.15.82:0/2009080529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe80c005540 con 0x7fe8381451e0 2024-03-16T17:57:48.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.891+0000 7fe834c00640 1 -- 172.21.15.82:0/2009080529 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe828015020 con 0x7fe8381451e0 2024-03-16T17:57:48.895 INFO:teuthology.orchestra.run.smithi082.stdout:60129542147 2024-03-16T17:57:48.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.894+0000 7fe82e000640 1 -- 172.21.15.82:0/2009080529 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe81806b480 msgr2=0x7fe81806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.895+0000 7fe82e000640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe81806b480 0x7fe81806d940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe830035000 tx=0x7fe830030000 comp rx=0 tx=0).stop 2024-03-16T17:57:48.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.895+0000 7fe82e000640 1 -- 172.21.15.82:0/2009080529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 msgr2=0x7fe838158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.895+0000 7fe82e000640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 0x7fe838158880 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe82800b410 tx=0x7fe82800b8d0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.896+0000 7fe82e000640 1 -- 172.21.15.82:0/2009080529 shutdown_connections 2024-03-16T17:57:48.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.896+0000 7fe82e000640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe81806b480 0x7fe81806d940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.896+0000 7fe82e000640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe8381455c0 0x7fe838149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.896+0000 7fe82e000640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8381451e0 0x7fe838158880 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.896+0000 7fe82e000640 1 --2- 172.21.15.82:0/2009080529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8380b2ff0 0x7fe838072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.896+0000 7fe82e000640 1 -- 172.21.15.82:0/2009080529 >> 172.21.15.82:0/2009080529 conn(0x7fe8380b17b0 msgr2=0x7fe838146d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.896+0000 7fe82e000640 1 -- 172.21.15.82:0/2009080529 shutdown_connections 2024-03-16T17:57:48.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.896+0000 7fe82e000640 1 -- 172.21.15.82:0/2009080529 wait complete. 2024-03-16T17:57:48.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.919+0000 7fabcd400640 1 -- 172.21.15.82:0/3063360628 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fabc8073420 con 0x7fabc806db20 2024-03-16T17:57:48.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.920+0000 7fabbea00640 1 -- 172.21.15.82:0/3063360628 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fabb4014020 con 0x7fabc806db20 2024-03-16T17:57:48.923 INFO:teuthology.orchestra.run.smithi082.stdout:60129542149 2024-03-16T17:57:48.925 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.2 2024-03-16T17:57:48.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.923+0000 7fabcd400640 1 -- 172.21.15.82:0/3063360628 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faba806b3b0 msgr2=0x7faba806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.923+0000 7fabcd400640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faba806b3b0 0x7faba806d870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabc8153d30 tx=0x7fabb8009040 comp rx=0 tx=0).stop 2024-03-16T17:57:48.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.923+0000 7fabcd400640 1 -- 172.21.15.82:0/3063360628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabc806db20 msgr2=0x7fabc81557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.923+0000 7fabcd400640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabc806db20 0x7fabc81557b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fabb400d5a0 tx=0x7fabb4001e70 comp rx=0 tx=0).stop 2024-03-16T17:57:48.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.923+0000 7fabcd400640 1 -- 172.21.15.82:0/3063360628 shutdown_connections 2024-03-16T17:57:48.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.923+0000 7fabcd400640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faba806b3b0 0x7faba806d870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.923+0000 7fabcd400640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fabc8140460 0x7fabc8154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.923+0000 7fabcd400640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabc80ccf10 0x7fabc8153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.923+0000 7fabcd400640 1 --2- 172.21.15.82:0/3063360628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabc806db20 0x7fabc81557b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.923+0000 7fabcd400640 1 -- 172.21.15.82:0/3063360628 >> 172.21.15.82:0/3063360628 conn(0x7fabc806ae70 msgr2=0x7fabc80cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.924+0000 7fabcd400640 1 -- 172.21.15.82:0/3063360628 shutdown_connections 2024-03-16T17:57:48.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.924+0000 7fabcd400640 1 -- 172.21.15.82:0/3063360628 wait complete. 2024-03-16T17:57:48.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.928+0000 7fd1fc800640 1 -- 172.21.15.82:0/3671938089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd1f40677c0 con 0x7fd1f40ad850 2024-03-16T17:57:48.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.930+0000 7fd1ee000640 1 -- 172.21.15.82:0/3671938089 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd1e4021020 con 0x7fd1f40ad850 2024-03-16T17:57:48.933 INFO:teuthology.orchestra.run.smithi082.stdout:60129542149 2024-03-16T17:57:48.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.930+0000 7f3bcb200640 1 -- 172.21.15.82:0/1417815671 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3bc4155ba0 con 0x7f3bc40b2d90 2024-03-16T17:57:48.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.931+0000 7f3bbbe00640 1 -- 172.21.15.82:0/1417815671 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3bc4155ba0 con 0x7f3bc40b2d90 2024-03-16T17:57:48.936 INFO:teuthology.orchestra.run.smithi082.stdout:60129542147 2024-03-16T17:57:48.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.933+0000 7fd1fc800640 1 -- 172.21.15.82:0/3671938089 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd1d806b3b0 msgr2=0x7fd1d806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.933+0000 7fd1fc800640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd1d806b3b0 0x7fd1d806d870 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8006fd0 tx=0x7fd1e8008040 comp rx=0 tx=0).stop 2024-03-16T17:57:48.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.933+0000 7fd1fc800640 1 -- 172.21.15.82:0/3671938089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f40ad850 msgr2=0x7fd1f4158860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.933+0000 7fd1fc800640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f40ad850 0x7fd1f4158860 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40010e0 tx=0x7fd1e400c960 comp rx=0 tx=0).stop 2024-03-16T17:57:48.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.934+0000 7fd1fc800640 1 -- 172.21.15.82:0/3671938089 shutdown_connections 2024-03-16T17:57:48.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.934+0000 7fd1fc800640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd1d806b3b0 0x7fd1d806d870 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.934+0000 7fd1fc800640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd1f41613b0 0x7fd1f415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.934+0000 7fd1fc800640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd1f40ccc50 0x7fd1f414e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.934+0000 7fd1fc800640 1 --2- 172.21.15.82:0/3671938089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1f40ad850 0x7fd1f4158860 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.934+0000 7fd1fc800640 1 -- 172.21.15.82:0/3671938089 >> 172.21.15.82:0/3671938089 conn(0x7fd1f406ae70 msgr2=0x7fd1f4071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.934+0000 7fd1fc800640 1 -- 172.21.15.82:0/3671938089 shutdown_connections 2024-03-16T17:57:48.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.935+0000 7fd1fc800640 1 -- 172.21.15.82:0/3671938089 wait complete. 2024-03-16T17:57:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.935+0000 7f3bcb200640 1 -- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3ba406b3b0 msgr2=0x7f3ba406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.935+0000 7f3bcb200640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3ba406b3b0 0x7f3ba406d870 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4001380 tx=0x7f3bb4009040 comp rx=0 tx=0).stop 2024-03-16T17:57:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.935+0000 7f3bcb200640 1 -- 172.21.15.82:0/1417815671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bc40b2d90 msgr2=0x7f3bc414ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.935+0000 7f3bcb200640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bc40b2d90 0x7f3bc414ff60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000d5f0 tx=0x7f3bb000dab0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.936+0000 7f3bcb200640 1 -- 172.21.15.82:0/1417815671 shutdown_connections 2024-03-16T17:57:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.936+0000 7f3bcb200640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3ba406b3b0 0x7f3ba406d870 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.936+0000 7f3bcb200640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3bc40cc800 0x7f3bc41504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.936+0000 7f3bcb200640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3bc40b2d90 0x7f3bc414ff60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.936+0000 7f3bcb200640 1 --2- 172.21.15.82:0/1417815671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bc406b960 0x7f3bc40b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.936+0000 7f3bcb200640 1 -- 172.21.15.82:0/1417815671 >> 172.21.15.82:0/1417815671 conn(0x7f3bc406ae70 msgr2=0x7f3bc406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.936+0000 7f3bcb200640 1 -- 172.21.15.82:0/1417815671 shutdown_connections 2024-03-16T17:57:48.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.936+0000 7f3bcb200640 1 -- 172.21.15.82:0/1417815671 wait complete. 2024-03-16T17:57:48.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.945+0000 7fcfcda00640 1 -- 172.21.15.82:0/3992933351 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcfc804f330 con 0x7fcfc80d1f60 2024-03-16T17:57:48.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.946+0000 7fcfbea00640 1 -- 172.21.15.82:0/3992933351 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcfc007b650 con 0x7fcfc80d1f60 2024-03-16T17:57:48.949 INFO:teuthology.orchestra.run.smithi082.stdout:60129542148 2024-03-16T17:57:48.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.948+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1094200c40 con 0x7f1094146240 2024-03-16T17:57:48.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.949+0000 7f1090c00640 1 -- 172.21.15.82:0/3990588651 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1094200c40 con 0x7f1094146240 2024-03-16T17:57:48.953 INFO:teuthology.orchestra.run.smithi082.stdout:60129542147 2024-03-16T17:57:48.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.6 2024-03-16T17:57:48.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.950+0000 7fcfcda00640 1 -- 172.21.15.82:0/3992933351 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfa806b3e0 msgr2=0x7fcfa806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.950+0000 7fcfcda00640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfa806b3e0 0x7fcfa806d8a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcfc81fb360 tx=0x7fcfb8008040 comp rx=0 tx=0).stop 2024-03-16T17:57:48.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.950+0000 7fcfcda00640 1 -- 172.21.15.82:0/3992933351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc80d1f60 msgr2=0x7fcfc8161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.950+0000 7fcfcda00640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc80d1f60 0x7fcfc8161080 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009e50 tx=0x7fcfc0004a40 comp rx=0 tx=0).stop 2024-03-16T17:57:48.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.950+0000 7fcfcda00640 1 -- 172.21.15.82:0/3992933351 shutdown_connections 2024-03-16T17:57:48.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.950+0000 7fcfcda00640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcfa806b3e0 0x7fcfa806d8a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.950+0000 7fcfcda00640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfc815de90 0x7fcfc80d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.950+0000 7fcfcda00640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcfc80d2340 0x7fcfc80d0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.950+0000 7fcfcda00640 1 --2- 172.21.15.82:0/3992933351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc80d1f60 0x7fcfc8161080 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.950+0000 7fcfcda00640 1 -- 172.21.15.82:0/3992933351 >> 172.21.15.82:0/3992933351 conn(0x7fcfc80d3df0 msgr2=0x7fcfc80cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7fcfcda00640 1 -- 172.21.15.82:0/3992933351 shutdown_connections 2024-03-16T17:57:48.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7fcfcda00640 1 -- 172.21.15.82:0/3992933351 wait complete. 2024-03-16T17:57:48.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f107406b3b0 msgr2=0x7f107406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f107406b3b0 0x7f107406d870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1080009b30 tx=0x7f1080009240 comp rx=0 tx=0).stop 2024-03-16T17:57:48.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1094146240 msgr2=0x7f10941466f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1094146240 0x7f10941466f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f108400c480 tx=0x7f108400c940 comp rx=0 tx=0).stop 2024-03-16T17:57:48.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 shutdown_connections 2024-03-16T17:57:48.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f107406b3b0 0x7f107406d870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1094146240 0x7f10941466f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10940b44f0 0x7f10941441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 --2- 172.21.15.82:0/3990588651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f10940aed50 0x7f1094158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 >> 172.21.15.82:0/3990588651 conn(0x7f109406ae70 msgr2=0x7f10940ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 shutdown_connections 2024-03-16T17:57:48.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.952+0000 7f109ba00640 1 -- 172.21.15.82:0/3990588651 wait complete. 2024-03-16T17:57:48.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.954+0000 7fa95d800640 1 -- 172.21.15.82:0/3689841466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa95814a9b0 con 0x7fa9581455c0 2024-03-16T17:57:48.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.955+0000 7fa94ea00640 1 -- 172.21.15.82:0/3689841466 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa9500587e0 con 0x7fa9581455c0 2024-03-16T17:57:48.959 INFO:teuthology.orchestra.run.smithi082.stdout:60129542148 2024-03-16T17:57:48.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.957+0000 7f8223a00640 1 -- 172.21.15.82:0/3100874279 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f81f4005810 con 0x7f821c14de20 2024-03-16T17:57:48.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7f8218c00640 1 -- 172.21.15.82:0/3100874279 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8208057e40 con 0x7f821c14de20 2024-03-16T17:57:48.961 INFO:teuthology.orchestra.run.smithi082.stdout:60129542149 2024-03-16T17:57:48.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 -- 172.21.15.82:0/3689841466 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa93406b480 msgr2=0x7fa93406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa93406b480 0x7fa93406d940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa940000f60 tx=0x7fa9400079b0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 -- 172.21.15.82:0/3689841466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9581455c0 msgr2=0x7fa958072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9581455c0 0x7fa958072470 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa950007e60 tx=0x7fa95000e4b0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 -- 172.21.15.82:0/3689841466 shutdown_connections 2024-03-16T17:57:48.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa93406b480 0x7fa93406d940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9581455c0 0x7fa958072470 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa9581451e0 0x7fa958071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 --2- 172.21.15.82:0/3689841466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9580b31c0 0x7fa958138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 -- 172.21.15.82:0/3689841466 >> 172.21.15.82:0/3689841466 conn(0x7fa9580b17b0 msgr2=0x7fa9580b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 -- 172.21.15.82:0/3689841466 shutdown_connections 2024-03-16T17:57:48.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.958+0000 7fa95d800640 1 -- 172.21.15.82:0/3689841466 wait complete. 2024-03-16T17:57:48.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.961+0000 7f8223a00640 1 -- 172.21.15.82:0/3100874279 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81fc06b3e0 msgr2=0x7f81fc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.961+0000 7f8223a00640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81fc06b3e0 0x7f81fc06d8a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f820c000c00 tx=0x7f820c02f000 comp rx=0 tx=0).stop 2024-03-16T17:57:48.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.961+0000 7f8223a00640 1 -- 172.21.15.82:0/3100874279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f821c14de20 msgr2=0x7f821c14e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:48.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.961+0000 7f8223a00640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f821c14de20 0x7f821c14e2a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f820800f4e0 tx=0x7f820800f9a0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.961+0000 7f8223a00640 1 -- 172.21.15.82:0/3100874279 shutdown_connections 2024-03-16T17:57:48.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.961+0000 7f8223a00640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81fc06b3e0 0x7f81fc06d8a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.961+0000 7f8223a00640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f821c156180 0x7f821c156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.961+0000 7f8223a00640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f821c14de20 0x7f821c14e2a0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f820800f4e0 tx=0x7f820800f9a0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.961+0000 7f8223a00640 1 --2- 172.21.15.82:0/3100874279 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8200003970 0x7f821c0b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:48.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.961+0000 7f8223a00640 1 -- 172.21.15.82:0/3100874279 >> 172.21.15.82:0/3100874279 conn(0x7f821c0d3d80 msgr2=0x7f821c06b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:48.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.0 2024-03-16T17:57:48.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.963+0000 7f8223a00640 1 -- 172.21.15.82:0/3100874279 shutdown_connections 2024-03-16T17:57:48.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:48.963+0000 7f8223a00640 1 -- 172.21.15.82:0/3100874279 wait complete. 2024-03-16T17:57:48.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.4 2024-03-16T17:57:48.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.5 2024-03-16T17:57:48.985 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2024-03-16T17:57:48.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.1 2024-03-16T17:57:48.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.3 2024-03-16T17:57:49.925 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-16T17:57:49.955 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-16T17:57:49.967 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-16T17:57:49.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-16T17:57:49.978 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-16T17:57:49.986 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-16T17:57:49.986 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-16T17:57:49.988 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-16T17:57:50.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.185+0000 7fd570a00640 1 Processor -- start 2024-03-16T17:57:50.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.185+0000 7fd570a00640 1 -- start start 2024-03-16T17:57:50.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.188+0000 7fd570a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd564050180 0x7fd5640e0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.188+0000 7fd570a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5640df4e0 con 0x7fd5640df100 2024-03-16T17:57:50.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.188+0000 7fd570a00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5640535f0 con 0x7fd5640530a0 2024-03-16T17:57:50.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.188+0000 7fd570a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56404f840 con 0x7fd564050180 2024-03-16T17:57:50.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.189+0000 7fd569e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd564050180 0x7fd5640e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.189+0000 7fd56a800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd5640df100 0x7fd56404fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56246/0 (socket says 172.21.15.82:56246) 2024-03-16T17:57:50.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.189+0000 7fd56a800640 1 -- 172.21.15.82:0/3036079579 learned_addr learned my addr 172.21.15.82:0/3036079579 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:50.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.190+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2410238442 0 0) 0x7fd5640df4e0 con 0x7fd5640df100 2024-03-16T17:57:50.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.190+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd550003520 con 0x7fd5640df100 2024-03-16T17:57:50.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921165626 0 0) 0x7fd5640535f0 con 0x7fd5640530a0 2024-03-16T17:57:50.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5640df4e0 con 0x7fd5640530a0 2024-03-16T17:57:50.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4179956042 0 0) 0x7fd550003520 con 0x7fd5640df100 2024-03-16T17:57:50.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd564050180 msgr2=0x7fd5640e0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:50.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd564050180 0x7fd5640e0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 >> v1:172.21.15.90:6789/0 conn(0x7fd5640530a0 legacy=0x7fd564005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5640538d0 con 0x7fd5640df100 2024-03-16T17:57:50.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fd56c04faa0 con 0x7fd5640df100 2024-03-16T17:57:50.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 >> v1:172.21.15.82:6789/0 conn(0x7fd5640df100 legacy=0x7fd56404fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd550003b30 0x7fd550003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5500047c0 0x7fd550004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd550005180 0x7fd55000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56404f840 con 0x7fd550003b30 2024-03-16T17:57:50.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55000bec0 con 0x7fd550005180 2024-03-16T17:57:50.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.191+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55000c000 con 0x7fd5500047c0 2024-03-16T17:57:50.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.193+0000 7fd56b200640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd550005180 0x7fd55000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.193+0000 7fd569e00640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5500047c0 0x7fd550004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.194+0000 7fd56a800640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd550003b30 0x7fd550003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.194+0000 7fd56b200640 1 -- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5500047c0 msgr2=0x7fd550004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:50.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.194+0000 7fd56b200640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5500047c0 0x7fd550004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.194+0000 7fd56b200640 1 -- 172.21.15.82:0/3036079579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd550003b30 msgr2=0x7fd550003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:50.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.194+0000 7fd56b200640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd550003b30 0x7fd550003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.194+0000 7fd56b200640 1 -- 172.21.15.82:0/3036079579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5640538d0 con 0x7fd550005180 2024-03-16T17:57:50.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.195+0000 7fd56a800640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd550003b30 0x7fd550003f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:50.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.197+0000 7fd56b200640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd550005180 0x7fd55000b980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd5540018d0 tx=0x7fd5500057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b90ba4e772c2ec6b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.197+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5540052d0 con 0x7fd550005180 2024-03-16T17:57:50.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.197+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd554005430 con 0x7fd550005180 2024-03-16T17:57:50.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.197+0000 7fd569400640 1 -- 172.21.15.82:0/3036079579 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554003810 con 0x7fd550005180 2024-03-16T17:57:50.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.198+0000 7fd570a00640 1 -- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd550005180 msgr2=0x7fd55000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.198+0000 7fd570a00640 1 --2- 172.21.15.82:0/3036079579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd550005180 0x7fd55000b980 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd5540018d0 tx=0x7fd5500057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.199+0000 7fd56b200640 1 -- 172.21.15.82:0/3036079579 reap_dead start 2024-03-16T17:57:50.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.199+0000 7fd570a00640 1 -- 172.21.15.82:0/3036079579 shutdown_connections 2024-03-16T17:57:50.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.199+0000 7fd570a00640 1 -- 172.21.15.82:0/3036079579 >> 172.21.15.82:0/3036079579 conn(0x7fd564009fb0 msgr2=0x7fd56400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.200+0000 7fd570a00640 1 -- 172.21.15.82:0/3036079579 shutdown_connections 2024-03-16T17:57:50.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.200+0000 7fd570a00640 1 -- 172.21.15.82:0/3036079579 wait complete. 2024-03-16T17:57:50.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.203+0000 7fd570a00640 1 Processor -- start 2024-03-16T17:57:50.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.204+0000 7fd570a00640 1 -- start start 2024-03-16T17:57:50.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.206+0000 7fd570a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd550005180 0x7fd5640cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.206+0000 7fd570a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd56404bb90 0x7fd56404a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.206+0000 7fd570a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd56404c010 0x7fd56404a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.207+0000 7fd56a800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd550005180 0x7fd5640cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.207+0000 7fd56a800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd550005180 0x7fd5640cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60256/0 (socket says 172.21.15.82:60256) 2024-03-16T17:57:50.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.207+0000 7fd569e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd56404bb90 0x7fd56404a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.207+0000 7fd569e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd56404bb90 0x7fd56404a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:60486/0 (socket says 172.21.15.82:60486) 2024-03-16T17:57:50.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.208+0000 7fd56b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd56404c010 0x7fd56404a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.208+0000 7fd56a800640 1 -- 172.21.15.82:0/2725221313 learned_addr learned my addr 172.21.15.82:0/2725221313 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.208+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640cf800 con 0x7fd56404c010 2024-03-16T17:57:50.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.208+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56404d5a0 con 0x7fd56404bb90 2024-03-16T17:57:50.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.208+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56404c500 con 0x7fd550005180 2024-03-16T17:57:50.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.209+0000 7fd56a800640 1 -- 172.21.15.82:0/2725221313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd56404bb90 msgr2=0x7fd56404a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.210+0000 7fd56a800640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd56404bb90 0x7fd56404a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.210+0000 7fd56a800640 1 -- 172.21.15.82:0/2725221313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd56404c010 msgr2=0x7fd56404a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.210+0000 7fd56a800640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd56404c010 0x7fd56404a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.210+0000 7fd56a800640 1 -- 172.21.15.82:0/2725221313 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56404c7e0 con 0x7fd550005180 2024-03-16T17:57:50.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.210+0000 7fd56b200640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd56404c010 0x7fd56404a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:50.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.212+0000 7fd56a800640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd550005180 0x7fd5640cf2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd56c050b50 tx=0x7fd56c068b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.212+0000 7fd562a00640 1 -- 172.21.15.82:0/2725221313 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0916c0 con 0x7fd550005180 2024-03-16T17:57:50.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.213+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56404cac0 con 0x7fd550005180 2024-03-16T17:57:50.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.213+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56404d080 con 0x7fd550005180 2024-03-16T17:57:50.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.213+0000 7fd562a00640 1 -- 172.21.15.82:0/2725221313 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c091820 con 0x7fd550005180 2024-03-16T17:57:50.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.214+0000 7fd562a00640 1 -- 172.21.15.82:0/2725221313 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c091bd0 con 0x7fd550005180 2024-03-16T17:57:50.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.216+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd544005290 con 0x7fd550005180 2024-03-16T17:57:50.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.218+0000 7fd562a00640 1 -- 172.21.15.82:0/2725221313 <== mon.2 v2:172.21.15.90:3301/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fd56c091d30 con 0x7fd550005180 2024-03-16T17:57:50.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.219+0000 7fd562a00640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd54806b3e0 0x7fd54806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.220+0000 7fd562a00640 1 -- 172.21.15.82:0/2725221313 <== mon.2 v2:172.21.15.90:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd56c1098b0 con 0x7fd550005180 2024-03-16T17:57:50.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.221+0000 7fd569e00640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd54806b3e0 0x7fd54806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.222+0000 7fd569e00640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd54806b3e0 0x7fd54806d8a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd564048100 tx=0x7fd55c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.230+0000 7f5cfd400640 1 Processor -- start 2024-03-16T17:57:50.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.230+0000 7f5cfd400640 1 -- start start 2024-03-16T17:57:50.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.231+0000 7f5cfd400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5cf80b2e30 0x7f5cf80cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.231+0000 7f5cfd400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf80d4cb0 con 0x7f5cf815e190 2024-03-16T17:57:50.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.231+0000 7f5cfd400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf80d1f70 con 0x7f5cf80d28c0 2024-03-16T17:57:50.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.231+0000 7f5cfd400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf80d49d0 con 0x7f5cf80b2e30 2024-03-16T17:57:50.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.232+0000 7f5cf6800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5cf815e190 0x7f5cf80d21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56260/0 (socket says 172.21.15.82:56260) 2024-03-16T17:57:50.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.232+0000 7f5cf6800640 1 -- 172.21.15.82:0/4203194605 learned_addr learned my addr 172.21.15.82:0/4203194605 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:50.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.232+0000 7f5cf7200640 1 --2- 172.21.15.82:0/4203194605 >> v2:172.21.15.90:3300/0 conn(0x7f5cf80b2e30 0x7f5cf80cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.233+0000 7f5cf7200640 1 -- 172.21.15.82:0/4203194605 >> v1:172.21.15.90:6789/0 conn(0x7f5cf80d28c0 legacy=0x7f5cf80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.233+0000 7f5cf7200640 1 -- 172.21.15.82:0/4203194605 >> v1:172.21.15.82:6789/0 conn(0x7f5cf815e190 legacy=0x7f5cf80d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.233+0000 7f5cf7200640 1 -- 172.21.15.82:0/4203194605 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf80d1c90 con 0x7f5cf80b2e30 2024-03-16T17:57:50.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.235+0000 7fd562a00640 1 -- 172.21.15.82:0/2725221313 <== mon.2 v2:172.21.15.90:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd56c0d4520 con 0x7fd550005180 2024-03-16T17:57:50.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.235+0000 7f5cf7200640 1 --2- 172.21.15.82:0/4203194605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 0x7f5cf80cf0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009d90 tx=0x7f5cf002bc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddb62c7dcf70ca55 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.236+0000 7f5cf5400640 1 -- 172.21.15.82:0/4203194605 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf002c9c0 con 0x7f5cf80b2e30 2024-03-16T17:57:50.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.237+0000 7f5cf5400640 1 -- 172.21.15.82:0/4203194605 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf002cb20 con 0x7f5cf80b2e30 2024-03-16T17:57:50.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.237+0000 7f5cf5400640 1 -- 172.21.15.82:0/4203194605 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0034440 con 0x7f5cf80b2e30 2024-03-16T17:57:50.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.238+0000 7f5cfd400640 1 -- 172.21.15.82:0/4203194605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 msgr2=0x7f5cf80cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.238+0000 7f5cfd400640 1 --2- 172.21.15.82:0/4203194605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 0x7f5cf80cf0a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009d90 tx=0x7f5cf002bc90 comp rx=0 tx=0).stop 2024-03-16T17:57:50.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.238+0000 7fd5bca00640 1 Processor -- start 2024-03-16T17:57:50.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.238+0000 7f5cfd400640 1 -- 172.21.15.82:0/4203194605 shutdown_connections 2024-03-16T17:57:50.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.238+0000 7f5cfd400640 1 --2- 172.21.15.82:0/4203194605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 0x7f5cf80cf0a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.239+0000 7f5cfd400640 1 -- 172.21.15.82:0/4203194605 >> 172.21.15.82:0/4203194605 conn(0x7f5cf80d3d80 msgr2=0x7f5cf80d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.239+0000 7fd5bca00640 1 -- start start 2024-03-16T17:57:50.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.239+0000 7f5cfd400640 1 -- 172.21.15.82:0/4203194605 shutdown_connections 2024-03-16T17:57:50.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.240+0000 7f5cfd400640 1 -- 172.21.15.82:0/4203194605 wait complete. 2024-03-16T17:57:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.241+0000 7fd5bca00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd5b0005ee0 0x7fd5b0006320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.241+0000 7fd5bca00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b0006860 0x7fd5b0006cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.241+0000 7fd5bca00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b0054330 con 0x7fd5b00ead00 2024-03-16T17:57:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.241+0000 7fd5bca00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b0053720 con 0x7fd5b0005ee0 2024-03-16T17:57:50.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.241+0000 7fd5bca00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b0053a00 con 0x7fd5b0006860 2024-03-16T17:57:50.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.242+0000 7fd5b7e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b0006860 0x7fd5b0006cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.242+0000 7fd5b6a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd5b0005ee0 0x7fd5b0006320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.242+0000 7fd5b7e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b0006860 0x7fd5b0006cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60272/0 (socket says 172.21.15.82:60272) 2024-03-16T17:57:50.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.242+0000 7fd5b7e00640 1 -- 172.21.15.82:0/805929673 learned_addr learned my addr 172.21.15.82:0/805929673 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.242+0000 7fd5b6a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd5b0005ee0 0x7fd5b0006320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:60500/0 (socket says 172.21.15.82:60500) 2024-03-16T17:57:50.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.243+0000 7fd5b6000640 1 -- 172.21.15.82:0/805929673 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230029396 0 0) 0x7fd5b0054330 con 0x7fd5b00ead00 2024-03-16T17:57:50.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.243+0000 7fd5b6000640 1 -- 172.21.15.82:0/805929673 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a0003520 con 0x7fd5b00ead00 2024-03-16T17:57:50.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.243+0000 7f5cfd400640 1 Processor -- start 2024-03-16T17:57:50.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.244+0000 7f5cfd400640 1 -- start start 2024-03-16T17:57:50.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.244+0000 7fd5b6a00640 1 -- 172.21.15.82:0/805929673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b0006860 msgr2=0x7fd5b0006cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:50.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.244+0000 7fd5b6a00640 1 --2- 172.21.15.82:0/805929673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b0006860 0x7fd5b0006cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.244+0000 7fd5b6a00640 1 -- 172.21.15.82:0/805929673 >> v1:172.21.15.90:6789/0 conn(0x7fd5b00ead00 legacy=0x7fd5b004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.244+0000 7fd5b6a00640 1 -- 172.21.15.82:0/805929673 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b0053ce0 con 0x7fd5b0005ee0 2024-03-16T17:57:50.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.244+0000 7fd5b7e00640 1 --2- 172.21.15.82:0/805929673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b0006860 0x7fd5b0006cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:50.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.245+0000 7fd5b6a00640 1 --2- 172.21.15.82:0/805929673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5b0005ee0 0x7fd5b0006320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40097a0 tx=0x7fd5a402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c65f088df900e92 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.246+0000 7f5cfd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 0x7f5cf81583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.246+0000 7f5cfd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf80d28c0 0x7f5cf815fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.246+0000 7f5cfd400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5cf815e190 0x7f5cf80c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.246+0000 7f5cfd400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf815f620 con 0x7f5cf80d28c0 2024-03-16T17:57:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.246+0000 7f5cfd400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf8158930 con 0x7f5cf80b2e30 2024-03-16T17:57:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.246+0000 7f5cfd400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf815f2f0 con 0x7f5cf815e190 2024-03-16T17:57:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.247+0000 7f5cf6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 0x7f5cf81583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.247+0000 7f5cf6800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 0x7f5cf81583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:60516/0 (socket says 172.21.15.82:60516) 2024-03-16T17:57:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.247+0000 7f5cf6800640 1 -- 172.21.15.82:0/3633942955 learned_addr learned my addr 172.21.15.82:0/3633942955 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.247+0000 7f5cf5e00640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf80d28c0 0x7f5cf815fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.247+0000 7fd5b6000640 1 -- 172.21.15.82:0/805929673 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a402d040 con 0x7fd5b0005ee0 2024-03-16T17:57:50.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.247+0000 7f5cf7200640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5cf815e190 0x7f5cf80c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.248+0000 7fd5b6000640 1 -- 172.21.15.82:0/805929673 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a4031810 con 0x7fd5b0005ee0 2024-03-16T17:57:50.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.248+0000 7fd5b6000640 1 -- 172.21.15.82:0/805929673 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4031ba0 con 0x7fd5b0005ee0 2024-03-16T17:57:50.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.249+0000 7fd5bca00640 1 -- 172.21.15.82:0/805929673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5b0005ee0 msgr2=0x7fd5b0006320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.249+0000 7fd5bca00640 1 --2- 172.21.15.82:0/805929673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5b0005ee0 0x7fd5b0006320 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40097a0 tx=0x7fd5a402b8b0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.248+0000 7f5cf5e00640 1 -- 172.21.15.82:0/3633942955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5cf815e190 msgr2=0x7f5cf80c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.248+0000 7f5cf5e00640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5cf815e190 0x7f5cf80c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.248+0000 7f5cf5e00640 1 -- 172.21.15.82:0/3633942955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 msgr2=0x7f5cf81583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.248+0000 7f5cf5e00640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 0x7f5cf81583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.248+0000 7f5cf5e00640 1 -- 172.21.15.82:0/3633942955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0009a70 con 0x7f5cf80d28c0 2024-03-16T17:57:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.249+0000 7f5cf6800640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 0x7f5cf81583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.249+0000 7f5cf5e00640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf80d28c0 0x7f5cf815fab0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400c780 tx=0x7f5ce400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.250+0000 7fd5bca00640 1 -- 172.21.15.82:0/805929673 shutdown_connections 2024-03-16T17:57:50.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.250+0000 7fd5bca00640 1 --2- 172.21.15.82:0/805929673 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b0006860 0x7fd5b0006cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.250+0000 7fd5bca00640 1 --2- 172.21.15.82:0/805929673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5b0005ee0 0x7fd5b0006320 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.250+0000 7fd5bca00640 1 -- 172.21.15.82:0/805929673 >> 172.21.15.82:0/805929673 conn(0x7fd5b0009fb0 msgr2=0x7fd5b000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.250+0000 7fd5bca00640 1 -- 172.21.15.82:0/805929673 shutdown_connections 2024-03-16T17:57:50.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.250+0000 7fd5bca00640 1 -- 172.21.15.82:0/805929673 wait complete. 2024-03-16T17:57:50.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.250+0000 7f5ceea00640 1 -- 172.21.15.82:0/3633942955 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4005270 con 0x7f5cf80d28c0 2024-03-16T17:57:50.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.250+0000 7f5ceea00640 1 -- 172.21.15.82:0/3633942955 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce40053d0 con 0x7f5cf80d28c0 2024-03-16T17:57:50.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.250+0000 7f5ceea00640 1 -- 172.21.15.82:0/3633942955 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce400a880 con 0x7f5cf80d28c0 2024-03-16T17:57:50.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.251+0000 7f5cfd400640 1 -- 172.21.15.82:0/3633942955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf8161350 con 0x7f5cf80d28c0 2024-03-16T17:57:50.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.252+0000 7f5cfd400640 1 -- 172.21.15.82:0/3633942955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf81591d0 con 0x7f5cf80d28c0 2024-03-16T17:57:50.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.254+0000 7fd5bca00640 1 Processor -- start 2024-03-16T17:57:50.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.254+0000 7f5cfd400640 1 -- 172.21.15.82:0/3633942955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf80a1db0 con 0x7f5cf80d28c0 2024-03-16T17:57:50.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.254+0000 7fd5bca00640 1 -- start start 2024-03-16T17:57:50.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.256+0000 7fd5bca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5b0005ee0 0x7fd5b00df000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.256+0000 7fd5bca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b0006860 0x7fd5b017d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.256+0000 7fd5bca00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b00ead00 0x7fd5b0044da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.256+0000 7fd5bca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b00df540 con 0x7fd5b0006860 2024-03-16T17:57:50.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.256+0000 7fd5bca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b017d7c0 con 0x7fd5b0005ee0 2024-03-16T17:57:50.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.256+0000 7fd5bca00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b0046d30 con 0x7fd5b00ead00 2024-03-16T17:57:50.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.257+0000 7fd5b6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b0006860 0x7fd5b017d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.257+0000 7fd5b6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b0006860 0x7fd5b017d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35420/0 (socket says 172.21.15.82:35420) 2024-03-16T17:57:50.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.257+0000 7fd5b6a00640 1 -- 172.21.15.82:0/930407761 learned_addr learned my addr 172.21.15.82:0/930407761 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.257+0000 7fd5b7e00640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b00ead00 0x7fd5b0044da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.258+0000 7fd5b6a00640 1 -- 172.21.15.82:0/930407761 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b00ead00 msgr2=0x7fd5b0044da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.258+0000 7fd5b6a00640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b00ead00 0x7fd5b0044da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.258+0000 7fd5b6a00640 1 -- 172.21.15.82:0/930407761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5b0005ee0 msgr2=0x7fd5b00df000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-16T17:57:50.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.258+0000 7fd5b6a00640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5b0005ee0 0x7fd5b00df000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.258+0000 7fd5b6a00640 1 -- 172.21.15.82:0/930407761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b8050140 con 0x7fd5b0006860 2024-03-16T17:57:50.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.259+0000 7fd5b6a00640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b0006860 0x7fd5b017d280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4000c00 tx=0x7fd5a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.259+0000 7fb867200640 1 Processor -- start 2024-03-16T17:57:50.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.259+0000 7fb867200640 1 -- start start 2024-03-16T17:57:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.262+0000 7f5ceea00640 1 -- 172.21.15.82:0/3633942955 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f5ce400a9e0 con 0x7f5cf80d28c0 2024-03-16T17:57:50.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.262+0000 7fb867200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8680cebf0 0x7fb868070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.262+0000 7fb867200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8680ae0b0 con 0x7fb86815dcc0 2024-03-16T17:57:50.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.262+0000 7fb867200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8680cf140 con 0x7fb868161910 2024-03-16T17:57:50.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.262+0000 7fb865e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb86815dcc0 0x7fb8680b3390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56272/0 (socket says 172.21.15.82:56272) 2024-03-16T17:57:50.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.262+0000 7fb867200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680ad850 con 0x7fb8680cebf0 2024-03-16T17:57:50.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.262+0000 7fb865e00640 1 -- 172.21.15.82:0/3792415192 learned_addr learned my addr 172.21.15.82:0/3792415192 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:50.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.263+0000 7fb865400640 1 --2- 172.21.15.82:0/3792415192 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8680cebf0 0x7fb868070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.263+0000 7fb864a00640 1 -- 172.21.15.82:0/3792415192 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199817220 0 0) 0x7fb8680cf140 con 0x7fb868161910 2024-03-16T17:57:50.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.263+0000 7fb864a00640 1 -- 172.21.15.82:0/3792415192 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb84c003520 con 0x7fb868161910 2024-03-16T17:57:50.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.263+0000 7fb865400640 1 -- 172.21.15.82:0/3792415192 >> v1:172.21.15.90:6789/0 conn(0x7fb868161910 legacy=0x7fb8680ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.264+0000 7f5ceea00640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5cd806b400 0x7f5cd806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.264+0000 7f5ceea00640 1 -- 172.21.15.82:0/3633942955 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ce4089290 con 0x7f5cf80d28c0 2024-03-16T17:57:50.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.264+0000 7f5cf6800640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5cd806b400 0x7f5cd806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.263+0000 7fb865400640 1 -- 172.21.15.82:0/3792415192 >> v1:172.21.15.82:6789/0 conn(0x7fb86815dcc0 legacy=0x7fb8680b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.263+0000 7fb865400640 1 -- 172.21.15.82:0/3792415192 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8680ada60 con 0x7fb8680cebf0 2024-03-16T17:57:50.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.266+0000 7fd5ab400640 1 -- 172.21.15.82:0/930407761 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a403ac50 con 0x7fd5b0006860 2024-03-16T17:57:50.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.266+0000 7fd5ab400640 1 -- 172.21.15.82:0/930407761 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a403add0 con 0x7fd5b0006860 2024-03-16T17:57:50.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.266+0000 7fd5ab400640 1 -- 172.21.15.82:0/930407761 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4044630 con 0x7fd5b0006860 2024-03-16T17:57:50.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.266+0000 7fd5bca00640 1 -- 172.21.15.82:0/930407761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a4009400 con 0x7fd5b0006860 2024-03-16T17:57:50.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.266+0000 7fd5bca00640 1 -- 172.21.15.82:0/930407761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b00472f0 con 0x7fd5b0006860 2024-03-16T17:57:50.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.267+0000 7f5cf6800640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5cd806b400 0x7f5cd806d8c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005880 tx=0x7f5ce8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.267+0000 7f5ceea00640 1 -- 172.21.15.82:0/3633942955 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5ce4054050 con 0x7f5cf80d28c0 2024-03-16T17:57:50.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.268+0000 7fb865400640 1 --2- 172.21.15.82:0/3792415192 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8680cebf0 0x7fb868070a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb8580097d0 tx=0x7fb85802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=130fee90a9af3c79 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.268+0000 7fb864a00640 1 -- 172.21.15.82:0/3792415192 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858030610 con 0x7fb8680cebf0 2024-03-16T17:57:50.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.268+0000 7fb864a00640 1 -- 172.21.15.82:0/3792415192 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb858030770 con 0x7fb8680cebf0 2024-03-16T17:57:50.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.268+0000 7fb864a00640 1 -- 172.21.15.82:0/3792415192 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858030b00 con 0x7fb8680cebf0 2024-03-16T17:57:50.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.268+0000 7fd5bca00640 1 -- 172.21.15.82:0/930407761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b00052c0 con 0x7fd5b0006860 2024-03-16T17:57:50.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.269+0000 7fb867200640 1 -- 172.21.15.82:0/3792415192 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8680cebf0 msgr2=0x7fb868070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.269+0000 7fb867200640 1 --2- 172.21.15.82:0/3792415192 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8680cebf0 0x7fb868070a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb8580097d0 tx=0x7fb85802b960 comp rx=0 tx=0).stop 2024-03-16T17:57:50.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.270+0000 7fb867200640 1 -- 172.21.15.82:0/3792415192 shutdown_connections 2024-03-16T17:57:50.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.270+0000 7fb867200640 1 --2- 172.21.15.82:0/3792415192 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb8680cebf0 0x7fb868070a30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.270+0000 7fb867200640 1 -- 172.21.15.82:0/3792415192 >> 172.21.15.82:0/3792415192 conn(0x7fb86806ae70 msgr2=0x7fb86806b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.270+0000 7fd5ab400640 1 -- 172.21.15.82:0/930407761 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fd5a404d450 con 0x7fd5b0006860 2024-03-16T17:57:50.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.272+0000 7fd5ab400640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd59806b480 0x7fd59806d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.272+0000 7fb867200640 1 -- 172.21.15.82:0/3792415192 shutdown_connections 2024-03-16T17:57:50.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.272+0000 7fb867200640 1 -- 172.21.15.82:0/3792415192 wait complete. 2024-03-16T17:57:50.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.272+0000 7fd5b7400640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd59806b480 0x7fd59806d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.273+0000 7fd5ab400640 1 -- 172.21.15.82:0/930407761 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5a40abd90 con 0x7fd5b0006860 2024-03-16T17:57:50.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.274+0000 7fd5b7400640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd59806b480 0x7fd59806d940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac001570 tx=0x7fd5ac0078a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.275+0000 7fb867200640 1 Processor -- start 2024-03-16T17:57:50.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.275+0000 7fb867200640 1 -- start start 2024-03-16T17:57:50.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.277+0000 7fb867200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8680cebf0 0x7fb86815af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.277+0000 7fb867200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb86815dcc0 0x7fb8680c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.277+0000 7fb867200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb868161910 0x7fb86814b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.277+0000 7fb867200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8680c4df0 con 0x7fb868161910 2024-03-16T17:57:50.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.277+0000 7fb867200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86815b5b0 con 0x7fb8680cebf0 2024-03-16T17:57:50.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.277+0000 7fb867200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86815b890 con 0x7fb86815dcc0 2024-03-16T17:57:50.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.277+0000 7fb866800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb868161910 0x7fb86814b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.277+0000 7fb865e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8680cebf0 0x7fb86815af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.277+0000 7fb865e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8680cebf0 0x7fb86815af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:60532/0 (socket says 172.21.15.82:60532) 2024-03-16T17:57:50.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.278+0000 7fb865e00640 1 -- 172.21.15.82:0/1678651112 learned_addr learned my addr 172.21.15.82:0/1678651112 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.278+0000 7fb865e00640 1 -- 172.21.15.82:0/1678651112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb86815dcc0 msgr2=0x7fb8680c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.278+0000 7fb865e00640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb86815dcc0 0x7fb8680c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.279+0000 7fb865e00640 1 -- 172.21.15.82:0/1678651112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb868161910 msgr2=0x7fb86814b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.279+0000 7fb865e00640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb868161910 0x7fb86814b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.279+0000 7fb865e00640 1 -- 172.21.15.82:0/1678651112 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb858009400 con 0x7fb8680cebf0 2024-03-16T17:57:50.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.279+0000 7fb865e00640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8680cebf0 0x7fb86815af90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb86000cba0 tx=0x7fb86000a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.280+0000 7fb85e000640 1 -- 172.21.15.82:0/1678651112 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86000ad40 con 0x7fb8680cebf0 2024-03-16T17:57:50.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.280+0000 7fb85e000640 1 -- 172.21.15.82:0/1678651112 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb860012400 con 0x7fb8680cebf0 2024-03-16T17:57:50.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.280+0000 7fb85e000640 1 -- 172.21.15.82:0/1678651112 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86000aea0 con 0x7fb8680cebf0 2024-03-16T17:57:50.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.280+0000 7fb867200640 1 -- 172.21.15.82:0/1678651112 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86815bb70 con 0x7fb8680cebf0 2024-03-16T17:57:50.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.280+0000 7fb867200640 1 -- 172.21.15.82:0/1678651112 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86815c130 con 0x7fb8680cebf0 2024-03-16T17:57:50.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.282+0000 7fb867200640 1 -- 172.21.15.82:0/1678651112 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86804f250 con 0x7fb8680cebf0 2024-03-16T17:57:50.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.284+0000 7fd5ab400640 1 -- 172.21.15.82:0/930407761 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd5a4076940 con 0x7fd5b0006860 2024-03-16T17:57:50.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.284+0000 7fb85e000640 1 -- 172.21.15.82:0/1678651112 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fb860012770 con 0x7fb8680cebf0 2024-03-16T17:57:50.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.286+0000 7fb85e000640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb84806b3e0 0x7fb84806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.286+0000 7fb85e000640 1 -- 172.21.15.82:0/1678651112 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb860089be0 con 0x7fb8680cebf0 2024-03-16T17:57:50.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.287+0000 7fb865400640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb84806b3e0 0x7fb84806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.288+0000 7fb865400640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb84806b3e0 0x7fb84806d8a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb858036000 tx=0x7fb85803b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.297+0000 7fb85e000640 1 -- 172.21.15.82:0/1678651112 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb8600548d0 con 0x7fb8680cebf0 2024-03-16T17:57:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.317+0000 7f4a23c00640 1 Processor -- start 2024-03-16T17:57:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.317+0000 7f4a23c00640 1 -- start start 2024-03-16T17:57:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.319+0000 7f4a23c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 0x7f4a1c0cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.319+0000 7f4a23c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c0d4cb0 con 0x7f4a1c0d28c0 2024-03-16T17:57:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.319+0000 7f4a23c00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c0d1f70 con 0x7f4a1c15e190 2024-03-16T17:57:50.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.319+0000 7f4a23c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0d49d0 con 0x7f4a1c0b2e30 2024-03-16T17:57:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.320+0000 7f4a1be00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4a1c0d28c0 0x7f4a1c0ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56284/0 (socket says 172.21.15.82:56284) 2024-03-16T17:57:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.320+0000 7f4a1be00640 1 -- 172.21.15.82:0/3156250724 learned_addr learned my addr 172.21.15.82:0/3156250724 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.320+0000 7f4a21c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 0x7f4a1c0cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.321+0000 7f4a1b400640 1 -- 172.21.15.82:0/3156250724 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445015976 0 0) 0x7f4a1c0d1f70 con 0x7f4a1c15e190 2024-03-16T17:57:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7faf16a00640 1 Processor -- start 2024-03-16T17:57:50.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.321+0000 7f64e2a00640 1 Processor -- start 2024-03-16T17:57:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7f4a1b400640 1 -- 172.21.15.82:0/3156250724 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a00003520 con 0x7f4a1c15e190 2024-03-16T17:57:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7f4a1b400640 1 -- 172.21.15.82:0/3156250724 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430080635 0 0) 0x7f4a1c0d4cb0 con 0x7f4a1c0d28c0 2024-03-16T17:57:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7f4a1b400640 1 -- 172.21.15.82:0/3156250724 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a1c0d1f70 con 0x7f4a1c0d28c0 2024-03-16T17:57:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7f9dcbc00640 1 Processor -- start 2024-03-16T17:57:50.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7f4a21c00640 1 -- 172.21.15.82:0/3156250724 >> v1:172.21.15.90:6789/0 conn(0x7f4a1c15e190 legacy=0x7f4a1c0d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7f4a1b400640 1 -- 172.21.15.82:0/3156250724 <== mon.1 v1:172.21.15.90:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2216646939 0 0) 0x7f4a00003520 con 0x7f4a1c15e190 2024-03-16T17:57:50.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7f4a21c00640 1 -- 172.21.15.82:0/3156250724 >> v1:172.21.15.82:6789/0 conn(0x7f4a1c0d28c0 legacy=0x7f4a1c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7f4a21c00640 1 -- 172.21.15.82:0/3156250724 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0d1c90 con 0x7f4a1c0b2e30 2024-03-16T17:57:50.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7faf16a00640 1 -- start start 2024-03-16T17:57:50.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.324+0000 7faf16a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7faf1006db20 0x7faf100b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.324+0000 7faf16a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf10140840 con 0x7faf10140460 2024-03-16T17:57:50.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.324+0000 7faf16a00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf100b39e0 con 0x7faf100ccf10 2024-03-16T17:57:50.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.324+0000 7faf16a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1015e880 con 0x7faf1006db20 2024-03-16T17:57:50.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7f9dcbc00640 1 -- start start 2024-03-16T17:57:50.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.322+0000 7f64e2a00640 1 -- start start 2024-03-16T17:57:50.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.325+0000 7faf0fe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7faf10140460 0x7faf100cc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56292/0 (socket says 172.21.15.82:56292) 2024-03-16T17:57:50.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.325+0000 7faf0fe00640 1 -- 172.21.15.82:0/430684932 learned_addr learned my addr 172.21.15.82:0/430684932 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:50.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.325+0000 7faf14a00640 1 --2- 172.21.15.82:0/430684932 >> v2:172.21.15.90:3300/0 conn(0x7faf1006db20 0x7faf100b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.325+0000 7f9dcbc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc415e190 0x7f9dc40d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.324+0000 7f64e2a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f64dc06b580 0x7f64dc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.324+0000 7f64e2a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64dc0aee90 0x7f64dc159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.324+0000 7f64e2a00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64dc15e180 con 0x7f64dc06d9d0 2024-03-16T17:57:50.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.324+0000 7f64e2a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64dc15d950 con 0x7f64dc0aee90 2024-03-16T17:57:50.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.324+0000 7f64e2a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64dc071aa0 con 0x7f64dc06b580 2024-03-16T17:57:50.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.325+0000 7f4a21c00640 1 --2- 172.21.15.82:0/3156250724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 0x7f4a1c0cf0a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009dc0 tx=0x7f4a1402bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9516f7e9d9bb787 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.325+0000 7f9dcbc00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9dc40b2e30 0x7f9dc40d26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.325+0000 7f64db400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64dc0aee90 0x7f64dc159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.325+0000 7f64db400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64dc0aee90 0x7f64dc159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35442/0 (socket says 172.21.15.82:35442) 2024-03-16T17:57:50.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.325+0000 7f64db400640 1 -- 172.21.15.82:0/3302786877 learned_addr learned my addr 172.21.15.82:0/3302786877 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7faf0ea00640 1 -- 172.21.15.82:0/430684932 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657013860 0 0) 0x7faf10140840 con 0x7faf10140460 2024-03-16T17:57:50.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7faf0ea00640 1 -- 172.21.15.82:0/430684932 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faef4003520 con 0x7faf10140460 2024-03-16T17:57:50.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7faf14a00640 1 -- 172.21.15.82:0/430684932 >> v1:172.21.15.90:6789/0 conn(0x7faf100ccf10 legacy=0x7faf100b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:50.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7faf14a00640 1 -- 172.21.15.82:0/430684932 >> v1:172.21.15.82:6789/0 conn(0x7faf10140460 legacy=0x7faf100cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f64dbe00640 1 --2- 172.21.15.82:0/3302786877 >> v2:172.21.15.90:3300/0 conn(0x7f64dc06b580 0x7f64dc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f4a1b400640 1 -- 172.21.15.82:0/3156250724 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08001670 con 0x7f4a1c0b2e30 2024-03-16T17:57:50.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f9dc9200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc415e190 0x7f9dc40d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f4a1b400640 1 -- 172.21.15.82:0/3156250724 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1402c810 con 0x7f4a1c0b2e30 2024-03-16T17:57:50.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f64db400640 1 -- 172.21.15.82:0/3302786877 >> v2:172.21.15.90:3300/0 conn(0x7f64dc06b580 msgr2=0x7f64dc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:50.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f64db400640 1 --2- 172.21.15.82:0/3302786877 >> v2:172.21.15.90:3300/0 conn(0x7f64dc06b580 0x7f64dc0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f64db400640 1 -- 172.21.15.82:0/3302786877 >> v1:172.21.15.90:6789/0 conn(0x7f64dc06d9d0 legacy=0x7f64dc06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:50.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f64db400640 1 -- 172.21.15.82:0/3302786877 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc160ab0 con 0x7f64dc0aee90 2024-03-16T17:57:50.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f4a1b400640 1 -- 172.21.15.82:0/3156250724 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1402cba0 con 0x7f4a1c0b2e30 2024-03-16T17:57:50.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f9dcbc00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc40d4cb0 con 0x7f9dc40b0ff0 2024-03-16T17:57:50.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7f9dc9200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc415e190 0x7f9dc40d21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60330/0 (socket says 172.21.15.82:60330) 2024-03-16T17:57:50.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.327+0000 7f9dc9200640 1 -- 172.21.15.82:0/611626859 learned_addr learned my addr 172.21.15.82:0/611626859 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.327+0000 7f9dcbc00640 1 -- 172.21.15.82:0/611626859 --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc40d1f70 con 0x7f9dc40b2e30 2024-03-16T17:57:50.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.327+0000 7f9dcbc00640 1 -- 172.21.15.82:0/611626859 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40d49d0 con 0x7f9dc415e190 2024-03-16T17:57:50.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.326+0000 7faf14a00640 1 -- 172.21.15.82:0/430684932 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1015dc00 con 0x7faf1006db20 2024-03-16T17:57:50.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.327+0000 7f4a23c00640 1 -- 172.21.15.82:0/3156250724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 msgr2=0x7f4a1c0cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.327+0000 7f4a23c00640 1 --2- 172.21.15.82:0/3156250724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 0x7f4a1c0cf0a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009dc0 tx=0x7f4a1402bce0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f4a23c00640 1 -- 172.21.15.82:0/3156250724 shutdown_connections 2024-03-16T17:57:50.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f4a23c00640 1 --2- 172.21.15.82:0/3156250724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 0x7f4a1c0cf0a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f4a23c00640 1 -- 172.21.15.82:0/3156250724 >> 172.21.15.82:0/3156250724 conn(0x7f4a1c0d3d80 msgr2=0x7f4a1c0d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f4a23c00640 1 -- 172.21.15.82:0/3156250724 shutdown_connections 2024-03-16T17:57:50.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f64db400640 1 --2- 172.21.15.82:0/3302786877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 0x7f64dc159bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0cd520 tx=0x7f64c802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78a137fd5e73e85 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f9dc3e00640 1 --2- 172.21.15.82:0/611626859 >> v2:172.21.15.90:3300/0 conn(0x7f9dc40b2e30 0x7f9dc40d26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f9dc3400640 1 -- 172.21.15.82:0/611626859 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100658058 0 0) 0x7f9dc40d4cb0 con 0x7f9dc40b0ff0 2024-03-16T17:57:50.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f64daa00640 1 -- 172.21.15.82:0/3302786877 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c802f850 con 0x7f64dc0aee90 2024-03-16T17:57:50.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f9dc9200640 1 -- 172.21.15.82:0/611626859 >> v2:172.21.15.90:3300/0 conn(0x7f9dc40b2e30 msgr2=0x7f9dc40d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:50.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.329+0000 7faf14a00640 1 --2- 172.21.15.82:0/430684932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf1006db20 0x7faf100b34a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faf08009d90 tx=0x7faf0802beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b66344e9851a4b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f9dc9200640 1 --2- 172.21.15.82:0/611626859 >> v2:172.21.15.90:3300/0 conn(0x7f9dc40b2e30 0x7f9dc40d26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f9dc9200640 1 -- 172.21.15.82:0/611626859 >> v1:172.21.15.90:6789/0 conn(0x7f9dc40b0ff0 legacy=0x7f9dc40b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f9dc9200640 1 -- 172.21.15.82:0/611626859 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc40d1c90 con 0x7f9dc415e190 2024-03-16T17:57:50.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.329+0000 7f64daa00640 1 -- 172.21.15.82:0/3302786877 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c802f9b0 con 0x7f64dc0aee90 2024-03-16T17:57:50.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.329+0000 7faf0ea00640 1 -- 172.21.15.82:0/430684932 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0802cc60 con 0x7faf1006db20 2024-03-16T17:57:50.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.329+0000 7faf0ea00640 1 -- 172.21.15.82:0/430684932 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0802cdc0 con 0x7faf1006db20 2024-03-16T17:57:50.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.329+0000 7faf0ea00640 1 -- 172.21.15.82:0/430684932 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf08034660 con 0x7faf1006db20 2024-03-16T17:57:50.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.329+0000 7f64daa00640 1 -- 172.21.15.82:0/3302786877 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c802f850 con 0x7f64dc0aee90 2024-03-16T17:57:50.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.330+0000 7f9dc3e00640 1 --2- 172.21.15.82:0/611626859 >> v2:172.21.15.90:3300/0 conn(0x7f9dc40b2e30 0x7f9dc40d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:57:50.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.330+0000 7f64e2a00640 1 -- 172.21.15.82:0/3302786877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 msgr2=0x7f64dc159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.330+0000 7f9dc9200640 1 --2- 172.21.15.82:0/611626859 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc415e190 0x7f9dc40d21b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009870 tx=0x7f9db402b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a55ae39f4ee749e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.330+0000 7faf16a00640 1 -- 172.21.15.82:0/430684932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf1006db20 msgr2=0x7faf100b34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.330+0000 7faf16a00640 1 --2- 172.21.15.82:0/430684932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf1006db20 0x7faf100b34a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faf08009d90 tx=0x7faf0802beb0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.328+0000 7f4a23c00640 1 -- 172.21.15.82:0/3156250724 wait complete. 2024-03-16T17:57:50.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.331+0000 7faf16a00640 1 -- 172.21.15.82:0/430684932 shutdown_connections 2024-03-16T17:57:50.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.331+0000 7faf16a00640 1 --2- 172.21.15.82:0/430684932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf1006db20 0x7faf100b34a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.330+0000 7f4a23c00640 1 Processor -- start 2024-03-16T17:57:50.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.331+0000 7faf16a00640 1 -- 172.21.15.82:0/430684932 >> 172.21.15.82:0/430684932 conn(0x7faf1006ae70 msgr2=0x7faf1006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.331+0000 7f9dc3400640 1 -- 172.21.15.82:0/611626859 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db40317e0 con 0x7f9dc415e190 2024-03-16T17:57:50.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.331+0000 7f9dc3400640 1 -- 172.21.15.82:0/611626859 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db4031940 con 0x7f9dc415e190 2024-03-16T17:57:50.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.331+0000 7f4a23c00640 1 -- start start 2024-03-16T17:57:50.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.331+0000 7faf16a00640 1 -- 172.21.15.82:0/430684932 shutdown_connections 2024-03-16T17:57:50.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.331+0000 7faf16a00640 1 -- 172.21.15.82:0/430684932 wait complete. 2024-03-16T17:57:50.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.331+0000 7f64e2a00640 1 --2- 172.21.15.82:0/3302786877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 0x7f64dc159bf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0cd520 tx=0x7f64c802ba30 comp rx=0 tx=0).stop 2024-03-16T17:57:50.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.332+0000 7f64e2a00640 1 -- 172.21.15.82:0/3302786877 shutdown_connections 2024-03-16T17:57:50.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.332+0000 7f9dcbc00640 1 -- 172.21.15.82:0/611626859 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc415e190 msgr2=0x7f9dc40d21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.332+0000 7f9dcbc00640 1 --2- 172.21.15.82:0/611626859 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc415e190 0x7f9dc40d21b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009870 tx=0x7f9db402b880 comp rx=0 tx=0).stop 2024-03-16T17:57:50.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.332+0000 7f64e2a00640 1 --2- 172.21.15.82:0/3302786877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 0x7f64dc159bf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.332+0000 7f64e2a00640 1 --2- 172.21.15.82:0/3302786877 >> v2:172.21.15.90:3300/0 conn(0x7f64dc06b580 0x7f64dc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.332+0000 7f64e2a00640 1 -- 172.21.15.82:0/3302786877 >> 172.21.15.82:0/3302786877 conn(0x7f64dc067280 msgr2=0x7f64dc06ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.333+0000 7f64e2a00640 1 -- 172.21.15.82:0/3302786877 shutdown_connections 2024-03-16T17:57:50.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.333+0000 7f4a23c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 0x7f4a1c0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.333+0000 7f4a23c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c0d28c0 0x7f4a1c15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.333+0000 7f4a23c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a1c15e190 0x7f4a1c0c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.334+0000 7faf16a00640 1 Processor -- start 2024-03-16T17:57:50.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.333+0000 7f64e2a00640 1 -- 172.21.15.82:0/3302786877 wait complete. 2024-03-16T17:57:50.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.334+0000 7faf16a00640 1 -- start start 2024-03-16T17:57:50.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.334+0000 7f4a21c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a1c15e190 0x7f4a1c0c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.334+0000 7f4a21c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a1c15e190 0x7f4a1c0c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:60574/0 (socket says 172.21.15.82:60574) 2024-03-16T17:57:50.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.334+0000 7f4a21c00640 1 -- 172.21.15.82:0/1589362343 learned_addr learned my addr 172.21.15.82:0/1589362343 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.334+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c15bd00 con 0x7f4a1c0d28c0 2024-03-16T17:57:50.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.334+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0cd3c0 con 0x7f4a1c15e190 2024-03-16T17:57:50.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f4a21200640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 0x7f4a1c0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c1f61e0 con 0x7f4a1c0b2e30 2024-03-16T17:57:50.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f9dcbc00640 1 -- 172.21.15.82:0/611626859 shutdown_connections 2024-03-16T17:57:50.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f9dcbc00640 1 --2- 172.21.15.82:0/611626859 >> v2:172.21.15.90:3300/0 conn(0x7f9dc40b2e30 0x7f9dc40d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f9dcbc00640 1 --2- 172.21.15.82:0/611626859 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc415e190 0x7f9dc40d21b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f9dcbc00640 1 -- 172.21.15.82:0/611626859 >> 172.21.15.82:0/611626859 conn(0x7f9dc40d3d80 msgr2=0x7f9dc40d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f4a1be00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c0d28c0 0x7f4a1c15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f4a21c00640 1 -- 172.21.15.82:0/1589362343 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 msgr2=0x7f4a1c0cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f4a21c00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 0x7f4a1c0cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f4a21c00640 1 -- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c0d28c0 msgr2=0x7f4a1c15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f4a21c00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c0d28c0 0x7f4a1c15ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f4a21c00640 1 -- 172.21.15.82:0/1589362343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a14009a70 con 0x7f4a1c15e190 2024-03-16T17:57:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.335+0000 7f4a1be00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c0d28c0 0x7f4a1c15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.336+0000 7f9dcbc00640 1 -- 172.21.15.82:0/611626859 shutdown_connections 2024-03-16T17:57:50.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.336+0000 7f9dcbc00640 1 -- 172.21.15.82:0/611626859 wait complete. 2024-03-16T17:57:50.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.336+0000 7f4a21c00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a1c15e190 0x7f4a1c0c78b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009ee0 tx=0x7f4a14004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.336+0000 7faf16a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1006db20 0x7faf1013b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7f64e2a00640 1 Processor -- start 2024-03-16T17:57:50.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7faf16a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf100ccf10 0x7faf10071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7faf16a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf10140460 0x7faf10071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7f64e2a00640 1 -- start start 2024-03-16T17:57:50.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7f4a18c00640 1 -- 172.21.15.82:0/1589362343 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14009450 con 0x7f4a1c15e190 2024-03-16T17:57:50.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7f4a18c00640 1 -- 172.21.15.82:0/1589362343 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a14034c50 con 0x7f4a1c15e190 2024-03-16T17:57:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7faf16a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf10155a30 con 0x7faf1006db20 2024-03-16T17:57:50.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7f4a18c00640 1 -- 172.21.15.82:0/1589362343 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a14040450 con 0x7f4a1c15e190 2024-03-16T17:57:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7faf0f400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf100ccf10 0x7faf10071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7faf0f400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf100ccf10 0x7faf10071990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60354/0 (socket says 172.21.15.82:60354) 2024-03-16T17:57:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7faf0f400640 1 -- 172.21.15.82:0/3310539338 learned_addr learned my addr 172.21.15.82:0/3310539338 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7faf16a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf10154e10 con 0x7faf10140460 2024-03-16T17:57:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.337+0000 7faf14a00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf10140460 0x7faf10071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.338+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1013b7e0 con 0x7faf100ccf10 2024-03-16T17:57:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.338+0000 7faf0fe00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1006db20 0x7faf1013b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.338+0000 7f64e2a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64dc06b580 0x7f64dc15faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.338+0000 7f64e2a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64dc06d9d0 0x7f64dc15a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.338+0000 7f64e2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 0x7f64dc15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.338+0000 7f64e2a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc15f660 con 0x7f64dc0aee90 2024-03-16T17:57:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.338+0000 7f64e2a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc147e80 con 0x7f64dc06b580 2024-03-16T17:57:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.338+0000 7f64e2a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc15e770 con 0x7f64dc06d9d0 2024-03-16T17:57:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.338+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c0cd660 con 0x7f4a1c15e190 2024-03-16T17:57:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.338+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c1f6480 con 0x7f4a1c15e190 2024-03-16T17:57:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7faf14a00640 1 -- 172.21.15.82:0/3310539338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf100ccf10 msgr2=0x7faf10071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7faf14a00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf100ccf10 0x7faf10071990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7faf14a00640 1 -- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1006db20 msgr2=0x7faf1013b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f64e0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 0x7f64dc15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f9dcbc00640 1 Processor -- start 2024-03-16T17:57:50.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f64dbe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64dc06b580 0x7f64dc15faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f9dcbc00640 1 -- start start 2024-03-16T17:57:50.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f64e0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 0x7f64dc15ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35468/0 (socket says 172.21.15.82:35468) 2024-03-16T17:57:50.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f64e0a00640 1 -- 172.21.15.82:0/54662121 learned_addr learned my addr 172.21.15.82:0/54662121 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f64db400640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64dc06d9d0 0x7f64dc15a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f64e0a00640 1 -- 172.21.15.82:0/54662121 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64dc06d9d0 msgr2=0x7f64dc15a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f64e0a00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64dc06d9d0 0x7f64dc15a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f64e0a00640 1 -- 172.21.15.82:0/54662121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64dc06b580 msgr2=0x7f64dc15faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f64e0a00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64dc06b580 0x7f64dc15faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7f64e0a00640 1 -- 172.21.15.82:0/54662121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c80094d0 con 0x7f64dc0aee90 2024-03-16T17:57:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.339+0000 7faf14a00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1006db20 0x7faf1013b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.340+0000 7f64e0a00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 0x7f64dc15ab80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00c3c0 tx=0x7f64cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.340+0000 7faf14a00640 1 -- 172.21.15.82:0/3310539338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf08009a70 con 0x7faf10140460 2024-03-16T17:57:50.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.340+0000 7f64d3e00640 1 -- 172.21.15.82:0/54662121 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0052d0 con 0x7f64dc0aee90 2024-03-16T17:57:50.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.340+0000 7faf0fe00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1006db20 0x7faf1013b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:50.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.340+0000 7f64d3e00640 1 -- 172.21.15.82:0/54662121 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc005430 con 0x7f64dc0aee90 2024-03-16T17:57:50.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.340+0000 7f64d3e00640 1 -- 172.21.15.82:0/54662121 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc007260 con 0x7f64dc0aee90 2024-03-16T17:57:50.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7faf14a00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf10140460 0x7faf10071ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf08033000 tx=0x7faf08004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dcbc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc40b0ff0 0x7f9dc4160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dcbc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc40b2e30 0x7f9dc41f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dcbc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc415e190 0x7f9dc41f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dcbc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc41f4c10 con 0x7f9dc40b2e30 2024-03-16T17:57:50.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dcbc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc4159b20 con 0x7f9dc415e190 2024-03-16T17:57:50.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dcbc00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40cd310 con 0x7f9dc40b0ff0 2024-03-16T17:57:50.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dc3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc40b2e30 0x7f9dc41f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dc9c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc415e190 0x7f9dc41f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dc3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc40b2e30 0x7f9dc41f7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35476/0 (socket says 172.21.15.82:35476) 2024-03-16T17:57:50.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dc3e00640 1 -- 172.21.15.82:0/2222563920 learned_addr learned my addr 172.21.15.82:0/2222563920 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:50.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7faf07e00640 1 -- 172.21.15.82:0/3310539338 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0803f420 con 0x7faf10140460 2024-03-16T17:57:50.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.341+0000 7f9dc9200640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc40b0ff0 0x7f9dc4160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.342+0000 7faf07e00640 1 -- 172.21.15.82:0/3310539338 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0803f580 con 0x7faf10140460 2024-03-16T17:57:50.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.342+0000 7f4a18c00640 1 -- 172.21.15.82:0/1589362343 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f4a140405b0 con 0x7f4a1c15e190 2024-03-16T17:57:50.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.343+0000 7f64e2a00640 1 -- 172.21.15.82:0/54662121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64dc15ea50 con 0x7f64dc0aee90 2024-03-16T17:57:50.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.342+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf10155090 con 0x7faf10140460 2024-03-16T17:57:50.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.343+0000 7f64e2a00640 1 -- 172.21.15.82:0/54662121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64dc15f010 con 0x7f64dc0aee90 2024-03-16T17:57:50.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.342+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf10155650 con 0x7faf10140460 2024-03-16T17:57:50.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.343+0000 7faf07e00640 1 -- 172.21.15.82:0/3310539338 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0803f8f0 con 0x7faf10140460 2024-03-16T17:57:50.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.343+0000 7f9dc3e00640 1 -- 172.21.15.82:0/2222563920 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc40b0ff0 msgr2=0x7f9dc4160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.343+0000 7f9dc3e00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc40b0ff0 0x7f9dc4160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.343+0000 7f9dc3e00640 1 -- 172.21.15.82:0/2222563920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc415e190 msgr2=0x7f9dc41f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.343+0000 7f9dc3e00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc415e190 0x7f9dc41f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.343+0000 7f9dc3e00640 1 -- 172.21.15.82:0/2222563920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db0009400 con 0x7f9dc40b2e30 2024-03-16T17:57:50.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.344+0000 7f9dc9200640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc40b0ff0 0x7f9dc4160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:50.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.344+0000 7f9dc9c00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc415e190 0x7f9dc41f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:50.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.343+0000 7f4a18c00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f49fc06b400 0x7f49fc06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.343+0000 7f4a18c00640 1 -- 172.21.15.82:0/1589362343 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a140ac620 con 0x7f4a1c15e190 2024-03-16T17:57:50.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.345+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c0d17a0 con 0x7f4a1c15e190 2024-03-16T17:57:50.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.345+0000 7f4a21200640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f49fc06b400 0x7f49fc06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.345+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee8005290 con 0x7faf10140460 2024-03-16T17:57:50.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.346+0000 7f9dc3e00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc40b2e30 0x7f9dc41f7040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9db000e980 tx=0x7f9db000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.346+0000 7f9dc0c00640 1 -- 172.21.15.82:0/2222563920 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00096e0 con 0x7f9dc40b2e30 2024-03-16T17:57:50.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.347+0000 7f4a21200640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f49fc06b400 0x7f49fc06d8c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a08007b00 tx=0x7f4a08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.345+0000 7f64e2a00640 1 -- 172.21.15.82:0/54662121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc15f7d0 con 0x7f64dc0aee90 2024-03-16T17:57:50.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.346+0000 7f64d3e00640 1 -- 172.21.15.82:0/54662121 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f64cc0073c0 con 0x7f64dc0aee90 2024-03-16T17:57:50.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.347+0000 7f9dc0c00640 1 -- 172.21.15.82:0/2222563920 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db000ca00 con 0x7f9dc40b2e30 2024-03-16T17:57:50.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.347+0000 7f9dc0c00640 1 -- 172.21.15.82:0/2222563920 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db000cd90 con 0x7f9dc40b2e30 2024-03-16T17:57:50.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.347+0000 7f9dcbc00640 1 -- 172.21.15.82:0/2222563920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db40094d0 con 0x7f9dc40b2e30 2024-03-16T17:57:50.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.347+0000 7f9dcbc00640 1 -- 172.21.15.82:0/2222563920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc40cd030 con 0x7f9dc40b2e30 2024-03-16T17:57:50.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.347+0000 7faf07e00640 1 -- 172.21.15.82:0/3310539338 <== mon.1 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7faf0803fa50 con 0x7faf10140460 2024-03-16T17:57:50.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.348+0000 7f64d3e00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f64bc06b400 0x7f64bc06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.348+0000 7f64d3e00640 1 -- 172.21.15.82:0/54662121 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64cc088e00 con 0x7f64dc0aee90 2024-03-16T17:57:50.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.349+0000 7faf07e00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faef006b3b0 0x7faef006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.349+0000 7f9dcbc00640 1 -- 172.21.15.82:0/2222563920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc40a1db0 con 0x7f9dc40b2e30 2024-03-16T17:57:50.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.349+0000 7f64dbe00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f64bc06b400 0x7f64bc06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.350+0000 7faf0fe00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faef006b3b0 0x7faef006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.350+0000 7faf07e00640 1 -- 172.21.15.82:0/3310539338 <== mon.1 v2:172.21.15.90:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf0807bbd0 con 0x7faf10140460 2024-03-16T17:57:50.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.350+0000 7f64dbe00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f64bc06b400 0x7f64bc06d8c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f64dc06b400 tx=0x7f64d40079a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.351+0000 7faf0fe00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faef006b3b0 0x7faef006d870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf00005880 tx=0x7faf00009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.353+0000 7f64d3e00640 1 -- 172.21.15.82:0/54662121 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f64cc053bf0 con 0x7f64dc0aee90 2024-03-16T17:57:50.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.357+0000 7f9dc0c00640 1 -- 172.21.15.82:0/2222563920 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f9db0013470 con 0x7f9dc40b2e30 2024-03-16T17:57:50.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.359+0000 7f9dc0c00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9da406b480 0x7f9da406d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:50.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.359+0000 7f9dc0c00640 1 -- 172.21.15.82:0/2222563920 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9db008a680 con 0x7f9dc40b2e30 2024-03-16T17:57:50.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.359+0000 7f9dc0c00640 1 -- 172.21.15.82:0/2222563920 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9db008aa60 con 0x7f9dc40b2e30 2024-03-16T17:57:50.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.360+0000 7f4a18c00640 1 -- 172.21.15.82:0/1589362343 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4a140771e0 con 0x7f4a1c15e190 2024-03-16T17:57:50.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.360+0000 7f9dc9200640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9da406b480 0x7f9da406d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:50.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.367+0000 7f9dc9200640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9da406b480 0x7f9da406d940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9db4036000 tx=0x7f9db403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:50.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.374+0000 7faf07e00640 1 -- 172.21.15.82:0/3310539338 <== mon.1 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf08077f00 con 0x7faf10140460 2024-03-16T17:57:50.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.657+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd544005540 con 0x7fd550005180 2024-03-16T17:57:50.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.661+0000 7fd562a00640 1 -- 172.21.15.82:0/2725221313 <== mon.2 v2:172.21.15.90:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd56c0d7be0 con 0x7fd550005180 2024-03-16T17:57:50.666 INFO:teuthology.orchestra.run.smithi082.stdout:60129542149 2024-03-16T17:57:50.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.665+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd54806b3e0 msgr2=0x7fd54806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.665+0000 7fd570a00640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd54806b3e0 0x7fd54806d8a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd564048100 tx=0x7fd55c008040 comp rx=0 tx=0).stop 2024-03-16T17:57:50.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.665+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd550005180 msgr2=0x7fd5640cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.665+0000 7fd570a00640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd550005180 0x7fd5640cf2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd56c050b50 tx=0x7fd56c068b50 comp rx=0 tx=0).stop 2024-03-16T17:57:50.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.665+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 shutdown_connections 2024-03-16T17:57:50.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.665+0000 7fd570a00640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd54806b3e0 0x7fd54806d8a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.665+0000 7fd570a00640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd56404c010 0x7fd56404a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.665+0000 7fd570a00640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd56404bb90 0x7fd56404a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.665+0000 7fd570a00640 1 --2- 172.21.15.82:0/2725221313 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd550005180 0x7fd5640cf2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.666+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 >> 172.21.15.82:0/2725221313 conn(0x7fd564009fb0 msgr2=0x7fd56404eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.667+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 shutdown_connections 2024-03-16T17:57:50.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.667+0000 7fd570a00640 1 -- 172.21.15.82:0/2725221313 wait complete. 2024-03-16T17:57:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.681+0000 7f5cfd400640 1 -- 172.21.15.82:0/3633942955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5cf80b5650 con 0x7f5cf80d28c0 2024-03-16T17:57:50.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.682+0000 7f5ceea00640 1 -- 172.21.15.82:0/3633942955 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ce4017090 con 0x7f5cf80d28c0 2024-03-16T17:57:50.686 INFO:teuthology.orchestra.run.smithi082.stdout:60129542151 2024-03-16T17:57:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.685+0000 7f5cfd400640 1 -- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5cd806b400 msgr2=0x7f5cd806d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.686+0000 7f5cfd400640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5cd806b400 0x7f5cd806d8c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8005880 tx=0x7f5ce8009040 comp rx=0 tx=0).stop 2024-03-16T17:57:50.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.686+0000 7f5cfd400640 1 -- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf80d28c0 msgr2=0x7f5cf815fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.686+0000 7f5cfd400640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf80d28c0 0x7f5cf815fab0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400c780 tx=0x7f5ce400cc40 comp rx=0 tx=0).stop 2024-03-16T17:57:50.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.687+0000 7f5cfd400640 1 -- 172.21.15.82:0/3633942955 shutdown_connections 2024-03-16T17:57:50.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.687+0000 7f5cfd400640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5cd806b400 0x7f5cd806d8c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.687+0000 7f5cfd400640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5cf815e190 0x7f5cf80c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.687+0000 7f5cfd400640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5cf80d28c0 0x7f5cf815fab0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.687+0000 7f5cfd400640 1 --2- 172.21.15.82:0/3633942955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5cf80b2e30 0x7f5cf81583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.687+0000 7f5cfd400640 1 -- 172.21.15.82:0/3633942955 >> 172.21.15.82:0/3633942955 conn(0x7f5cf80d3d80 msgr2=0x7f5cf806b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.687+0000 7f5cfd400640 1 -- 172.21.15.82:0/3633942955 shutdown_connections 2024-03-16T17:57:50.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.688+0000 7f5cfd400640 1 -- 172.21.15.82:0/3633942955 wait complete. 2024-03-16T17:57:50.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.2 2024-03-16T17:57:50.697 DEBUG:teuthology.parallel:result is None 2024-03-16T17:57:50.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.703+0000 7fb867200640 1 -- 172.21.15.82:0/1678651112 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb8680cdd50 con 0x7fb8680cebf0 2024-03-16T17:57:50.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.704+0000 7fb85e000640 1 -- 172.21.15.82:0/1678651112 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb860017090 con 0x7fb8680cebf0 2024-03-16T17:57:50.707 INFO:teuthology.orchestra.run.smithi082.stdout:60129542149 2024-03-16T17:57:50.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.706+0000 7fb867200640 1 -- 172.21.15.82:0/1678651112 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb84806b3e0 msgr2=0x7fb84806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.706+0000 7fb867200640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb84806b3e0 0x7fb84806d8a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb858036000 tx=0x7fb85803b040 comp rx=0 tx=0).stop 2024-03-16T17:57:50.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.706+0000 7fb867200640 1 -- 172.21.15.82:0/1678651112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8680cebf0 msgr2=0x7fb86815af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.706+0000 7fb867200640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8680cebf0 0x7fb86815af90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb86000cba0 tx=0x7fb86000a430 comp rx=0 tx=0).stop 2024-03-16T17:57:50.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.707+0000 7fb867200640 1 -- 172.21.15.82:0/1678651112 shutdown_connections 2024-03-16T17:57:50.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.707+0000 7fb867200640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb84806b3e0 0x7fb84806d8a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.707+0000 7fb867200640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb868161910 0x7fb86814b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.707+0000 7fb867200640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb86815dcc0 0x7fb8680c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.707+0000 7fb867200640 1 --2- 172.21.15.82:0/1678651112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8680cebf0 0x7fb86815af90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.707+0000 7fb867200640 1 -- 172.21.15.82:0/1678651112 >> 172.21.15.82:0/1678651112 conn(0x7fb86806ae70 msgr2=0x7fb8680b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.707+0000 7fb867200640 1 -- 172.21.15.82:0/1678651112 shutdown_connections 2024-03-16T17:57:50.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.707+0000 7fb867200640 1 -- 172.21.15.82:0/1678651112 wait complete. 2024-03-16T17:57:50.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.0 2024-03-16T17:57:50.716 DEBUG:teuthology.parallel:result is None 2024-03-16T17:57:50.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.715+0000 7fd5bca00640 1 -- 172.21.15.82:0/930407761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd5b000c2b0 con 0x7fd5b0006860 2024-03-16T17:57:50.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.716+0000 7fd5ab400640 1 -- 172.21.15.82:0/930407761 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd5a4035090 con 0x7fd5b0006860 2024-03-16T17:57:50.719 INFO:teuthology.orchestra.run.smithi082.stdout:60129542151 2024-03-16T17:57:50.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 -- 172.21.15.82:0/930407761 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd59806b480 msgr2=0x7fd59806d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd59806b480 0x7fd59806d940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac001570 tx=0x7fd5ac0078a0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 -- 172.21.15.82:0/930407761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b0006860 msgr2=0x7fd5b017d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b0006860 0x7fd5b017d280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4000c00 tx=0x7fd5a40046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 -- 172.21.15.82:0/930407761 shutdown_connections 2024-03-16T17:57:50.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd59806b480 0x7fd59806d940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd5b00ead00 0x7fd5b0044da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd5b0006860 0x7fd5b017d280 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 --2- 172.21.15.82:0/930407761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5b0005ee0 0x7fd5b00df000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 -- 172.21.15.82:0/930407761 >> 172.21.15.82:0/930407761 conn(0x7fd5b0009fb0 msgr2=0x7fd5b0052c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 -- 172.21.15.82:0/930407761 shutdown_connections 2024-03-16T17:57:50.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.721+0000 7fd5bca00640 1 -- 172.21.15.82:0/930407761 wait complete. 2024-03-16T17:57:50.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.4 2024-03-16T17:57:50.729 DEBUG:teuthology.parallel:result is None 2024-03-16T17:57:50.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.6 2024-03-16T17:57:50.742 DEBUG:teuthology.parallel:result is None 2024-03-16T17:57:50.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.766+0000 7f64e2a00640 1 -- 172.21.15.82:0/54662121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f64dc04f250 con 0x7f64dc0aee90 2024-03-16T17:57:50.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.767+0000 7f64d3e00640 1 -- 172.21.15.82:0/54662121 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f64cc0572b0 con 0x7f64dc0aee90 2024-03-16T17:57:50.770 INFO:teuthology.orchestra.run.smithi082.stdout:60129542150 2024-03-16T17:57:50.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 -- 172.21.15.82:0/54662121 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f64bc06b400 msgr2=0x7f64bc06d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f64bc06b400 0x7f64bc06d8c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f64dc06b400 tx=0x7f64d40079a0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 -- 172.21.15.82:0/54662121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 msgr2=0x7f64dc15ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 0x7f64dc15ab80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f64cc00c3c0 tx=0x7f64cc00c880 comp rx=0 tx=0).stop 2024-03-16T17:57:50.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 -- 172.21.15.82:0/54662121 shutdown_connections 2024-03-16T17:57:50.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f64bc06b400 0x7f64bc06d8c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64dc0aee90 0x7f64dc15ab80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f64dc06d9d0 0x7f64dc15a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 --2- 172.21.15.82:0/54662121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64dc06b580 0x7f64dc15faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 -- 172.21.15.82:0/54662121 >> 172.21.15.82:0/54662121 conn(0x7f64dc067280 msgr2=0x7f64dc071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 -- 172.21.15.82:0/54662121 shutdown_connections 2024-03-16T17:57:50.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.769+0000 7f64e2a00640 1 -- 172.21.15.82:0/54662121 wait complete. 2024-03-16T17:57:50.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.782+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7faee8005810 con 0x7faf10140460 2024-03-16T17:57:50.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.783+0000 7faf07e00640 1 -- 172.21.15.82:0/3310539338 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf08043020 con 0x7faf10140460 2024-03-16T17:57:50.786 INFO:teuthology.orchestra.run.smithi082.stdout:60129542151 2024-03-16T17:57:50.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faef006b3b0 msgr2=0x7faef006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faef006b3b0 0x7faef006d870 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf00005880 tx=0x7faf00009040 comp rx=0 tx=0).stop 2024-03-16T17:57:50.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf10140460 msgr2=0x7faf10071ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf10140460 0x7faf10071ed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf08033000 tx=0x7faf08004a40 comp rx=0 tx=0).stop 2024-03-16T17:57:50.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 shutdown_connections 2024-03-16T17:57:50.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faef006b3b0 0x7faef006d870 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf10140460 0x7faf10071ed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf100ccf10 0x7faf10071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 --2- 172.21.15.82:0/3310539338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf1006db20 0x7faf1013b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 >> 172.21.15.82:0/3310539338 conn(0x7faf1006ae70 msgr2=0x7faf1013d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 shutdown_connections 2024-03-16T17:57:50.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.785+0000 7faf16a00640 1 -- 172.21.15.82:0/3310539338 wait complete. 2024-03-16T17:57:50.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.3 2024-03-16T17:57:50.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.787+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4a1c0a1db0 con 0x7f4a1c15e190 2024-03-16T17:57:50.791 DEBUG:teuthology.parallel:result is None 2024-03-16T17:57:50.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.788+0000 7f4a18c00640 1 -- 172.21.15.82:0/1589362343 <== mon.1 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4a14039020 con 0x7f4a1c15e190 2024-03-16T17:57:50.791 INFO:teuthology.orchestra.run.smithi082.stdout:60129542149 2024-03-16T17:57:50.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.791+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f49fc06b400 msgr2=0x7f49fc06d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.791+0000 7f4a23c00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f49fc06b400 0x7f49fc06d8c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a08007b00 tx=0x7f4a08009040 comp rx=0 tx=0).stop 2024-03-16T17:57:50.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.791+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a1c15e190 msgr2=0x7f4a1c0c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.791+0000 7f4a23c00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a1c15e190 0x7f4a1c0c78b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009ee0 tx=0x7f4a14004a40 comp rx=0 tx=0).stop 2024-03-16T17:57:50.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.792+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 shutdown_connections 2024-03-16T17:57:50.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.792+0000 7f4a23c00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f49fc06b400 0x7f49fc06d8c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.792+0000 7f4a23c00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4a1c15e190 0x7f4a1c0c78b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.792+0000 7f4a23c00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4a1c0d28c0 0x7f4a1c15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.792+0000 7f4a23c00640 1 --2- 172.21.15.82:0/1589362343 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4a1c0b2e30 0x7f4a1c0cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.792+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 >> 172.21.15.82:0/1589362343 conn(0x7f4a1c0d3d80 msgr2=0x7f4a1c0ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.792+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 shutdown_connections 2024-03-16T17:57:50.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.792+0000 7f4a23c00640 1 -- 172.21.15.82:0/1589362343 wait complete. 2024-03-16T17:57:50.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.795+0000 7f9dcbc00640 1 -- 172.21.15.82:0/2222563920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9dc40b5650 con 0x7f9dc40b2e30 2024-03-16T17:57:50.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.795+0000 7f9dc0c00640 1 -- 172.21.15.82:0/2222563920 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9db0055350 con 0x7f9dc40b2e30 2024-03-16T17:57:50.799 INFO:teuthology.orchestra.run.smithi082.stdout:60129542150 2024-03-16T17:57:50.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.797+0000 7f9dcbc00640 1 -- 172.21.15.82:0/2222563920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9da406b480 msgr2=0x7f9da406d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.797+0000 7f9dcbc00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9da406b480 0x7f9da406d940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9db4036000 tx=0x7f9db403d040 comp rx=0 tx=0).stop 2024-03-16T17:57:50.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.797+0000 7f9dcbc00640 1 -- 172.21.15.82:0/2222563920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc40b2e30 msgr2=0x7f9dc41f7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:50.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.797+0000 7f9dcbc00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc40b2e30 0x7f9dc41f7040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9db000e980 tx=0x7f9db000ee40 comp rx=0 tx=0).stop 2024-03-16T17:57:50.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.797+0000 7f9dcbc00640 1 -- 172.21.15.82:0/2222563920 shutdown_connections 2024-03-16T17:57:50.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.797+0000 7f9dcbc00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9da406b480 0x7f9da406d940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.797+0000 7f9dcbc00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc415e190 0x7f9dc41f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.797+0000 7f9dcbc00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9dc40b2e30 0x7f9dc41f7040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.797+0000 7f9dcbc00640 1 --2- 172.21.15.82:0/2222563920 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9dc40b0ff0 0x7f9dc4160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.797+0000 7f9dcbc00640 1 -- 172.21.15.82:0/2222563920 >> 172.21.15.82:0/2222563920 conn(0x7f9dc40d3d80 msgr2=0x7f9dc406b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.798+0000 7f9dcbc00640 1 -- 172.21.15.82:0/2222563920 shutdown_connections 2024-03-16T17:57:50.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:50.798+0000 7f9dcbc00640 1 -- 172.21.15.82:0/2222563920 wait complete. 2024-03-16T17:57:50.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.7 2024-03-16T17:57:50.810 DEBUG:teuthology.parallel:result is None 2024-03-16T17:57:50.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.5 2024-03-16T17:57:50.813 DEBUG:teuthology.parallel:result is None 2024-03-16T17:57:50.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.1 2024-03-16T17:57:50.818 DEBUG:teuthology.parallel:result is None 2024-03-16T17:57:50.818 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-16T17:57:50.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T17:57:51.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.046+0000 7f6f55000640 1 Processor -- start 2024-03-16T17:57:51.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.047+0000 7f6f55000640 1 -- start start 2024-03-16T17:57:51.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.048+0000 7f6f55000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f500b1210 0x7f6f5006bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.048+0000 7f6f55000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6f5006c5b0 0x7f6f5006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.048+0000 7f6f55000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f500d1140 con 0x7f6f5015e080 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.048+0000 7f6f55000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f500cf3e0 con 0x7f6f500b1210 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.048+0000 7f6f55000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5006cf80 con 0x7f6f5006c5b0 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.048+0000 7f6f4de00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6f5015e080 0x7f6f500d0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56306/0 (socket says 172.21.15.82:56306) 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.048+0000 7f6f4de00640 1 -- 172.21.15.82:0/3779050730 learned_addr learned my addr 172.21.15.82:0/3779050730 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.048+0000 7f6f4d400640 1 --2- 172.21.15.82:0/3779050730 >> v2:172.21.15.82:3300/0 conn(0x7f6f500b1210 0x7f6f5006bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.049+0000 7f6f4e800640 1 --2- 172.21.15.82:0/3779050730 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6f5006c5b0 0x7f6f5006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.049+0000 7f6f4ca00640 1 -- 172.21.15.82:0/3779050730 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603535778 0 0) 0x7f6f500d1140 con 0x7f6f5015e080 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.049+0000 7f6f4ca00640 1 -- 172.21.15.82:0/3779050730 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f34003520 con 0x7f6f5015e080 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.049+0000 7f6f4d400640 1 -- 172.21.15.82:0/3779050730 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6f5006c5b0 msgr2=0x7f6f5006ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.049+0000 7f6f4d400640 1 --2- 172.21.15.82:0/3779050730 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6f5006c5b0 0x7f6f5006ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.049+0000 7f6f4d400640 1 -- 172.21.15.82:0/3779050730 >> v1:172.21.15.82:6789/0 conn(0x7f6f5015e080 legacy=0x7f6f500d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.049+0000 7f6f4d400640 1 -- 172.21.15.82:0/3779050730 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5015e460 con 0x7f6f500b1210 2024-03-16T17:57:51.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.050+0000 7f6f4d400640 1 --2- 172.21.15.82:0/3779050730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 0x7f6f5006bf80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6f380097d0 tx=0x7f6f3802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e769ce6189354c31 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.050+0000 7f6f4ca00640 1 -- 172.21.15.82:0/3779050730 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38031890 con 0x7f6f500b1210 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.050+0000 7f6f4ca00640 1 -- 172.21.15.82:0/3779050730 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f380319f0 con 0x7f6f500b1210 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.050+0000 7f6f4ca00640 1 -- 172.21.15.82:0/3779050730 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38031d80 con 0x7f6f500b1210 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.051+0000 7f6f55000640 1 -- 172.21.15.82:0/3779050730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 msgr2=0x7f6f5006bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.051+0000 7f6f55000640 1 --2- 172.21.15.82:0/3779050730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 0x7f6f5006bf80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6f380097d0 tx=0x7f6f3802b930 comp rx=0 tx=0).stop 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.051+0000 7f6f55000640 1 -- 172.21.15.82:0/3779050730 shutdown_connections 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.051+0000 7f6f55000640 1 --2- 172.21.15.82:0/3779050730 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6f5006c5b0 0x7f6f5006ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.051+0000 7f6f55000640 1 --2- 172.21.15.82:0/3779050730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 0x7f6f5006bf80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.051+0000 7f6f55000640 1 -- 172.21.15.82:0/3779050730 >> 172.21.15.82:0/3779050730 conn(0x7f6f500b3780 msgr2=0x7f6f500b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.051+0000 7f6f55000640 1 -- 172.21.15.82:0/3779050730 shutdown_connections 2024-03-16T17:57:51.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.051+0000 7f6f55000640 1 -- 172.21.15.82:0/3779050730 wait complete. 2024-03-16T17:57:51.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.053+0000 7f6f55000640 1 Processor -- start 2024-03-16T17:57:51.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.053+0000 7f6f55000640 1 -- start start 2024-03-16T17:57:51.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.053+0000 7f6f55000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f5006c5b0 0x7f6f5015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.053+0000 7f6f55000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 0x7f6f500b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f55000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6f5015e080 0x7f6f500b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f55000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500cb640 con 0x7f6f500b1210 2024-03-16T17:57:51.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f55000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5015ccf0 con 0x7f6f5006c5b0 2024-03-16T17:57:51.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f55000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500ca9c0 con 0x7f6f5015e080 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 0x7f6f500b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f4d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 0x7f6f500b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35492/0 (socket says 172.21.15.82:35492) 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f4d400640 1 -- 172.21.15.82:0/4003424263 learned_addr learned my addr 172.21.15.82:0/4003424263 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f4d400640 1 -- 172.21.15.82:0/4003424263 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6f5015e080 msgr2=0x7f6f500b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f4d400640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6f5015e080 0x7f6f500b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f4d400640 1 -- 172.21.15.82:0/4003424263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f5006c5b0 msgr2=0x7f6f5015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f4d400640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f5006c5b0 0x7f6f5015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f4d400640 1 -- 172.21.15.82:0/4003424263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f38009400 con 0x7f6f500b1210 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.054+0000 7f6f4d400640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 0x7f6f500b49c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6f3802bf40 tx=0x7f6f380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.055+0000 7f6f46000640 1 -- 172.21.15.82:0/4003424263 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3803bc70 con 0x7f6f500b1210 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.055+0000 7f6f46000640 1 -- 172.21.15.82:0/4003424263 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3803bdf0 con 0x7f6f500b1210 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.055+0000 7f6f55000640 1 -- 172.21.15.82:0/4003424263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f500caca0 con 0x7f6f500b1210 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.055+0000 7f6f46000640 1 -- 172.21.15.82:0/4003424263 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38045630 con 0x7f6f500b1210 2024-03-16T17:57:51.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.055+0000 7f6f55000640 1 -- 172.21.15.82:0/4003424263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f500cb260 con 0x7f6f500b1210 2024-03-16T17:57:51.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.056+0000 7f6f55000640 1 -- 172.21.15.82:0/4003424263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f50067aa0 con 0x7f6f500b1210 2024-03-16T17:57:51.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.057+0000 7f6f46000640 1 -- 172.21.15.82:0/4003424263 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f6f38045790 con 0x7f6f500b1210 2024-03-16T17:57:51.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.057+0000 7f6f46000640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6f2c06b3b0 0x7f6f2c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.057+0000 7f6f46000640 1 -- 172.21.15.82:0/4003424263 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f380ac420 con 0x7f6f500b1210 2024-03-16T17:57:51.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.058+0000 7f6f4de00640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6f2c06b3b0 0x7f6f2c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:51.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.058+0000 7f6f4de00640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6f2c06b3b0 0x7f6f2c06d870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f40000f60 tx=0x7f6f400078f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:51.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.062+0000 7f6f46000640 1 -- 172.21.15.82:0/4003424263 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6f38076fe0 con 0x7f6f500b1210 2024-03-16T17:57:51.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.241+0000 7f6f55000640 1 -- 172.21.15.82:0/4003424263 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f500d0230 con 0x7f6f2c06b3b0 2024-03-16T17:57:51.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.259+0000 7f6f46000640 1 -- 172.21.15.82:0/4003424263 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251838 (secure 0 0 0) 0x7f6f40005890 con 0x7f6f2c06b3b0 2024-03-16T17:57:51.267 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:51.268 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T17:57:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.265+0000 7f6f55000640 1 -- 172.21.15.82:0/4003424263 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6f2c06b3b0 msgr2=0x7f6f2c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.265+0000 7f6f55000640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6f2c06b3b0 0x7f6f2c06d870 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f40000f60 tx=0x7f6f400078f0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.265+0000 7f6f55000640 1 -- 172.21.15.82:0/4003424263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 msgr2=0x7f6f500b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.265+0000 7f6f55000640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 0x7f6f500b49c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6f3802bf40 tx=0x7f6f380046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.266+0000 7f6f55000640 1 -- 172.21.15.82:0/4003424263 shutdown_connections 2024-03-16T17:57:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.266+0000 7f6f55000640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6f2c06b3b0 0x7f6f2c06d870 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.266+0000 7f6f55000640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6f5015e080 0x7f6f500b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.266+0000 7f6f55000640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f500b1210 0x7f6f500b49c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.266+0000 7f6f55000640 1 --2- 172.21.15.82:0/4003424263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f5006c5b0 0x7f6f5015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.266+0000 7f6f55000640 1 -- 172.21.15.82:0/4003424263 >> 172.21.15.82:0/4003424263 conn(0x7f6f500b3780 msgr2=0x7f6f5015cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.266+0000 7f6f55000640 1 -- 172.21.15.82:0/4003424263 shutdown_connections 2024-03-16T17:57:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.267+0000 7f6f55000640 1 -- 172.21.15.82:0/4003424263 wait complete. 2024-03-16T17:57:51.286 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-16T17:57:50.013336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2957312,"data_stored":1778860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.651215"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736016+0000","last_change":"2024-03-16T17:57:41.412158+0000","last_active":"2024-03-16T17:57:42.736016+0000","last_peered":"2024-03-16T17:57:42.736016+0000","last_clean":"2024-03-16T17:57:42.736016+0000","last_became_active":"2024-03-16T17:57:41.411766+0000","last_became_peered":"2024-03-16T17:57:41.411766+0000","last_unstale":"2024-03-16T17:57:42.736016+0000","last_undegraded":"2024-03-16T17:57:42.736016+0000","last_fullsized":"2024-03-16T17:57:42.736016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:50:09.742348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370170+0000","last_change":"2024-03-16T17:57:41.383449+0000","last_active":"2024-03-16T17:57:42.370170+0000","last_peered":"2024-03-16T17:57:42.370170+0000","last_clean":"2024-03-16T17:57:42.370170+0000","last_became_active":"2024-03-16T17:57:41.382165+0000","last_became_peered":"2024-03-16T17:57:41.382165+0000","last_unstale":"2024-03-16T17:57:42.370170+0000","last_undegraded":"2024-03-16T17:57:42.370170+0000","last_fullsized":"2024-03-16T17:57:42.370170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732651+0000","last_change":"2024-03-16T17:57:41.380082+0000","last_active":"2024-03-16T17:57:42.732651+0000","last_peered":"2024-03-16T17:57:42.732651+0000","last_clean":"2024-03-16T17:57:42.732651+0000","last_became_active":"2024-03-16T17:57:41.379508+0000","last_became_peered":"2024-03-16T17:57:41.379508+0000","last_unstale":"2024-03-16T17:57:42.732651+0000","last_undegraded":"2024-03-16T17:57:42.732651+0000","last_fullsized":"2024-03-16T17:57:42.732651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375083+0000","last_change":"2024-03-16T17:57:41.404364+0000","last_active":"2024-03-16T17:57:42.375083+0000","last_peered":"2024-03-16T17:57:42.375083+0000","last_clean":"2024-03-16T17:57:42.375083+0000","last_became_active":"2024-03-16T17:57:41.404048+0000","last_became_peered":"2024-03-16T17:57:41.404048+0000","last_unstale":"2024-03-16T17:57:42.375083+0000","last_undegraded":"2024-03-16T17:57:42.375083+0000","last_fullsized":"2024-03-16T17:57:42.375083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734821+0000","last_change":"2024-03-16T17:57:41.392327+0000","last_active":"2024-03-16T17:57:42.734821+0000","last_peered":"2024-03-16T17:57:42.734821+0000","last_clean":"2024-03-16T17:57:42.734821+0000","last_became_active":"2024-03-16T17:57:41.392007+0000","last_became_peered":"2024-03-16T17:57:41.392007+0000","last_unstale":"2024-03-16T17:57:42.734821+0000","last_undegraded":"2024-03-16T17:57:42.734821+0000","last_fullsized":"2024-03-16T17:57:42.734821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:48:10.857666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375324+0000","last_change":"2024-03-16T17:57:41.398446+0000","last_active":"2024-03-16T17:57:42.375324+0000","last_peered":"2024-03-16T17:57:42.375324+0000","last_clean":"2024-03-16T17:57:42.375324+0000","last_became_active":"2024-03-16T17:57:41.398135+0000","last_became_peered":"2024-03-16T17:57:41.398135+0000","last_unstale":"2024-03-16T17:57:42.375324+0000","last_undegraded":"2024-03-16T17:57:42.375324+0000","last_fullsized":"2024-03-16T17:57:42.375324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:48:10.857666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734440+0000","last_change":"2024-03-16T17:57:41.405535+0000","last_active":"2024-03-16T17:57:42.734440+0000","last_peered":"2024-03-16T17:57:42.734440+0000","last_clean":"2024-03-16T17:57:42.734440+0000","last_became_active":"2024-03-16T17:57:41.405323+0000","last_became_peered":"2024-03-16T17:57:41.405323+0000","last_unstale":"2024-03-16T17:57:42.734440+0000","last_undegraded":"2024-03-16T17:57:42.734440+0000","last_fullsized":"2024-03-16T17:57:42.734440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.731940+0000","last_change":"2024-03-16T17:57:41.402459+0000","last_active":"2024-03-16T17:57:42.731940+0000","last_peered":"2024-03-16T17:57:42.731940+0000","last_clean":"2024-03-16T17:57:42.731940+0000","last_became_active":"2024-03-16T17:57:41.402103+0000","last_became_peered":"2024-03-16T17:57:41.402103+0000","last_unstale":"2024-03-16T17:57:42.731940+0000","last_undegraded":"2024-03-16T17:57:42.731940+0000","last_fullsized":"2024-03-16T17:57:42.731940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:20.628509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736292+0000","last_change":"2024-03-16T17:57:41.409011+0000","last_active":"2024-03-16T17:57:42.736292+0000","last_peered":"2024-03-16T17:57:42.736292+0000","last_clean":"2024-03-16T17:57:42.736292+0000","last_became_active":"2024-03-16T17:57:41.408811+0000","last_became_peered":"2024-03-16T17:57:41.408811+0000","last_unstale":"2024-03-16T17:57:42.736292+0000","last_undegraded":"2024-03-16T17:57:42.736292+0000","last_fullsized":"2024-03-16T17:57:42.736292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:11:48.511692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734591+0000","last_change":"2024-03-16T17:57:41.401707+0000","last_active":"2024-03-16T17:57:42.734591+0000","last_peered":"2024-03-16T17:57:42.734591+0000","last_clean":"2024-03-16T17:57:42.734591+0000","last_became_active":"2024-03-16T17:57:41.401330+0000","last_became_peered":"2024-03-16T17:57:41.401330+0000","last_unstale":"2024-03-16T17:57:42.734591+0000","last_undegraded":"2024-03-16T17:57:42.734591+0000","last_fullsized":"2024-03-16T17:57:42.734591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733619+0000","last_change":"2024-03-16T17:57:41.399956+0000","last_active":"2024-03-16T17:57:42.733619+0000","last_peered":"2024-03-16T17:57:42.733619+0000","last_clean":"2024-03-16T17:57:42.733619+0000","last_became_active":"2024-03-16T17:57:41.399696+0000","last_became_peered":"2024-03-16T17:57:41.399696+0000","last_unstale":"2024-03-16T17:57:42.733619+0000","last_undegraded":"2024-03-16T17:57:42.733619+0000","last_fullsized":"2024-03-16T17:57:42.733619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373739+0000","last_change":"2024-03-16T17:57:41.397224+0000","last_active":"2024-03-16T17:57:42.373739+0000","last_peered":"2024-03-16T17:57:42.373739+0000","last_clean":"2024-03-16T17:57:42.373739+0000","last_became_active":"2024-03-16T17:57:41.396857+0000","last_became_peered":"2024-03-16T17:57:41.396857+0000","last_unstale":"2024-03-16T17:57:42.373739+0000","last_undegraded":"2024-03-16T17:57:42.373739+0000","last_fullsized":"2024-03-16T17:57:42.373739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:32:52.837630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734251+0000","last_change":"2024-03-16T17:57:41.399774+0000","last_active":"2024-03-16T17:57:42.734251+0000","last_peered":"2024-03-16T17:57:42.734251+0000","last_clean":"2024-03-16T17:57:42.734251+0000","last_became_active":"2024-03-16T17:57:41.399505+0000","last_became_peered":"2024-03-16T17:57:41.399505+0000","last_unstale":"2024-03-16T17:57:42.734251+0000","last_undegraded":"2024-03-16T17:57:42.734251+0000","last_fullsized":"2024-03-16T17:57:42.734251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732264+0000","last_change":"2024-03-16T17:57:41.380132+0000","last_active":"2024-03-16T17:57:42.732264+0000","last_peered":"2024-03-16T17:57:42.732264+0000","last_clean":"2024-03-16T17:57:42.732264+0000","last_became_active":"2024-03-16T17:57:41.379536+0000","last_became_peered":"2024-03-16T17:57:41.379536+0000","last_unstale":"2024-03-16T17:57:42.732264+0000","last_undegraded":"2024-03-16T17:57:42.732264+0000","last_fullsized":"2024-03-16T17:57:42.732264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:54:00.468586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369395+0000","last_change":"2024-03-16T17:57:41.393322+0000","last_active":"2024-03-16T17:57:42.369395+0000","last_peered":"2024-03-16T17:57:42.369395+0000","last_clean":"2024-03-16T17:57:42.369395+0000","last_became_active":"2024-03-16T17:57:41.391272+0000","last_became_peered":"2024-03-16T17:57:41.391272+0000","last_unstale":"2024-03-16T17:57:42.369395+0000","last_undegraded":"2024-03-16T17:57:42.369395+0000","last_fullsized":"2024-03-16T17:57:42.369395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374104+0000","last_change":"2024-03-16T17:57:41.399992+0000","last_active":"2024-03-16T17:57:42.374104+0000","last_peered":"2024-03-16T17:57:42.374104+0000","last_clean":"2024-03-16T17:57:42.374104+0000","last_became_active":"2024-03-16T17:57:41.399406+0000","last_became_peered":"2024-03-16T17:57:41.399406+0000","last_unstale":"2024-03-16T17:57:42.374104+0000","last_undegraded":"2024-03-16T17:57:42.374104+0000","last_fullsized":"2024-03-16T17:57:42.374104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376607+0000","last_change":"2024-03-16T17:57:41.416986+0000","last_active":"2024-03-16T17:57:42.376607+0000","last_peered":"2024-03-16T17:57:42.376607+0000","last_clean":"2024-03-16T17:57:42.376607+0000","last_became_active":"2024-03-16T17:57:41.416699+0000","last_became_peered":"2024-03-16T17:57:41.416699+0000","last_unstale":"2024-03-16T17:57:42.376607+0000","last_undegraded":"2024-03-16T17:57:42.376607+0000","last_fullsized":"2024-03-16T17:57:42.376607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:11:48.511692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733913+0000","last_change":"2024-03-16T17:57:41.394311+0000","last_active":"2024-03-16T17:57:42.733913+0000","last_peered":"2024-03-16T17:57:42.733913+0000","last_clean":"2024-03-16T17:57:42.733913+0000","last_became_active":"2024-03-16T17:57:41.393750+0000","last_became_peered":"2024-03-16T17:57:41.393750+0000","last_unstale":"2024-03-16T17:57:42.733913+0000","last_undegraded":"2024-03-16T17:57:42.733913+0000","last_fullsized":"2024-03-16T17:57:42.733913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:55:54.818895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736015+0000","last_change":"2024-03-16T17:57:41.408696+0000","last_active":"2024-03-16T17:57:42.736015+0000","last_peered":"2024-03-16T17:57:42.736015+0000","last_clean":"2024-03-16T17:57:42.736015+0000","last_became_active":"2024-03-16T17:57:41.407916+0000","last_became_peered":"2024-03-16T17:57:41.407916+0000","last_unstale":"2024-03-16T17:57:42.736015+0000","last_undegraded":"2024-03-16T17:57:42.736015+0000","last_fullsized":"2024-03-16T17:57:42.736015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:36:37.396828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733788+0000","last_change":"2024-03-16T17:57:41.396339+0000","last_active":"2024-03-16T17:57:42.733788+0000","last_peered":"2024-03-16T17:57:42.733788+0000","last_clean":"2024-03-16T17:57:42.733788+0000","last_became_active":"2024-03-16T17:57:41.396068+0000","last_became_peered":"2024-03-16T17:57:41.396068+0000","last_unstale":"2024-03-16T17:57:42.733788+0000","last_undegraded":"2024-03-16T17:57:42.733788+0000","last_fullsized":"2024-03-16T17:57:42.733788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732607+0000","last_change":"2024-03-16T17:57:41.390555+0000","last_active":"2024-03-16T17:57:42.732607+0000","last_peered":"2024-03-16T17:57:42.732607+0000","last_clean":"2024-03-16T17:57:42.732607+0000","last_became_active":"2024-03-16T17:57:41.389886+0000","last_became_peered":"2024-03-16T17:57:41.389886+0000","last_unstale":"2024-03-16T17:57:42.732607+0000","last_undegraded":"2024-03-16T17:57:42.732607+0000","last_fullsized":"2024-03-16T17:57:42.732607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:20.628509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375544+0000","last_change":"2024-03-16T17:57:41.411237+0000","last_active":"2024-03-16T17:57:42.375544+0000","last_peered":"2024-03-16T17:57:42.375544+0000","last_clean":"2024-03-16T17:57:42.375544+0000","last_became_active":"2024-03-16T17:57:41.410757+0000","last_became_peered":"2024-03-16T17:57:41.410757+0000","last_unstale":"2024-03-16T17:57:42.375544+0000","last_undegraded":"2024-03-16T17:57:42.375544+0000","last_fullsized":"2024-03-16T17:57:42.375544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:01:51.467418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733707+0000","last_change":"2024-03-16T17:57:41.391770+0000","last_active":"2024-03-16T17:57:42.733707+0000","last_peered":"2024-03-16T17:57:42.733707+0000","last_clean":"2024-03-16T17:57:42.733707+0000","last_became_active":"2024-03-16T17:57:41.391470+0000","last_became_peered":"2024-03-16T17:57:41.391470+0000","last_unstale":"2024-03-16T17:57:42.733707+0000","last_undegraded":"2024-03-16T17:57:42.733707+0000","last_fullsized":"2024-03-16T17:57:42.733707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374329+0000","last_change":"2024-03-16T17:57:41.400595+0000","last_active":"2024-03-16T17:57:42.374329+0000","last_peered":"2024-03-16T17:57:42.374329+0000","last_clean":"2024-03-16T17:57:42.374329+0000","last_became_active":"2024-03-16T17:57:41.400295+0000","last_became_peered":"2024-03-16T17:57:41.400295+0000","last_unstale":"2024-03-16T17:57:42.374329+0000","last_undegraded":"2024-03-16T17:57:42.374329+0000","last_fullsized":"2024-03-16T17:57:42.374329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:55:42.207131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734977+0000","last_change":"2024-03-16T17:57:41.391182+0000","last_active":"2024-03-16T17:57:42.734977+0000","last_peered":"2024-03-16T17:57:42.734977+0000","last_clean":"2024-03-16T17:57:42.734977+0000","last_became_active":"2024-03-16T17:57:41.390811+0000","last_became_peered":"2024-03-16T17:57:41.390811+0000","last_unstale":"2024-03-16T17:57:42.734977+0000","last_undegraded":"2024-03-16T17:57:42.734977+0000","last_fullsized":"2024-03-16T17:57:42.734977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:55:42.207131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734433+0000","last_change":"2024-03-16T17:57:41.407541+0000","last_active":"2024-03-16T17:57:42.734433+0000","last_peered":"2024-03-16T17:57:42.734433+0000","last_clean":"2024-03-16T17:57:42.734433+0000","last_became_active":"2024-03-16T17:57:41.405546+0000","last_became_peered":"2024-03-16T17:57:41.405546+0000","last_unstale":"2024-03-16T17:57:42.734433+0000","last_undegraded":"2024-03-16T17:57:42.734433+0000","last_fullsized":"2024-03-16T17:57:42.734433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369831+0000","last_change":"2024-03-16T17:57:41.399134+0000","last_active":"2024-03-16T17:57:42.369831+0000","last_peered":"2024-03-16T17:57:42.369831+0000","last_clean":"2024-03-16T17:57:42.369831+0000","last_became_active":"2024-03-16T17:57:41.398841+0000","last_became_peered":"2024-03-16T17:57:41.398841+0000","last_unstale":"2024-03-16T17:57:42.369831+0000","last_undegraded":"2024-03-16T17:57:42.369831+0000","last_fullsized":"2024-03-16T17:57:42.369831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734641+0000","last_change":"2024-03-16T17:57:41.399114+0000","last_active":"2024-03-16T17:57:42.734641+0000","last_peered":"2024-03-16T17:57:42.734641+0000","last_clean":"2024-03-16T17:57:42.734641+0000","last_became_active":"2024-03-16T17:57:41.395668+0000","last_became_peered":"2024-03-16T17:57:41.395668+0000","last_unstale":"2024-03-16T17:57:42.734641+0000","last_undegraded":"2024-03-16T17:57:42.734641+0000","last_fullsized":"2024-03-16T17:57:42.734641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732560+0000","last_change":"2024-03-16T17:57:41.390678+0000","last_active":"2024-03-16T17:57:42.732560+0000","last_peered":"2024-03-16T17:57:42.732560+0000","last_clean":"2024-03-16T17:57:42.732560+0000","last_became_active":"2024-03-16T17:57:41.387968+0000","last_became_peered":"2024-03-16T17:57:41.387968+0000","last_unstale":"2024-03-16T17:57:42.732560+0000","last_undegraded":"2024-03-16T17:57:42.732560+0000","last_fullsized":"2024-03-16T17:57:42.732560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:59:12.277190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375711+0000","last_change":"2024-03-16T17:57:41.399083+0000","last_active":"2024-03-16T17:57:42.375711+0000","last_peered":"2024-03-16T17:57:42.375711+0000","last_clean":"2024-03-16T17:57:42.375711+0000","last_became_active":"2024-03-16T17:57:41.398899+0000","last_became_peered":"2024-03-16T17:57:41.398899+0000","last_unstale":"2024-03-16T17:57:42.375711+0000","last_undegraded":"2024-03-16T17:57:42.375711+0000","last_fullsized":"2024-03-16T17:57:42.375711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:10:39.722613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733367+0000","last_change":"2024-03-16T17:57:41.390700+0000","last_active":"2024-03-16T17:57:42.733367+0000","last_peered":"2024-03-16T17:57:42.733367+0000","last_clean":"2024-03-16T17:57:42.733367+0000","last_became_active":"2024-03-16T17:57:41.389732+0000","last_became_peered":"2024-03-16T17:57:41.389732+0000","last_unstale":"2024-03-16T17:57:42.733367+0000","last_undegraded":"2024-03-16T17:57:42.733367+0000","last_fullsized":"2024-03-16T17:57:42.733367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:32:52.837630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732990+0000","last_change":"2024-03-16T17:57:41.392698+0000","last_active":"2024-03-16T17:57:42.732990+0000","last_peered":"2024-03-16T17:57:42.732990+0000","last_clean":"2024-03-16T17:57:42.732990+0000","last_became_active":"2024-03-16T17:57:41.391950+0000","last_became_peered":"2024-03-16T17:57:41.391950+0000","last_unstale":"2024-03-16T17:57:42.732990+0000","last_undegraded":"2024-03-16T17:57:42.732990+0000","last_fullsized":"2024-03-16T17:57:42.732990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.372924+0000","last_change":"2024-03-16T17:57:41.383046+0000","last_active":"2024-03-16T17:57:42.372924+0000","last_peered":"2024-03-16T17:57:42.372924+0000","last_clean":"2024-03-16T17:57:42.372924+0000","last_became_active":"2024-03-16T17:57:41.382533+0000","last_became_peered":"2024-03-16T17:57:41.382533+0000","last_unstale":"2024-03-16T17:57:42.372924+0000","last_undegraded":"2024-03-16T17:57:42.372924+0000","last_fullsized":"2024-03-16T17:57:42.372924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:44:11.757790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734496+0000","last_change":"2024-03-16T17:57:40.368183+0000","last_active":"2024-03-16T17:57:42.734496+0000","last_peered":"2024-03-16T17:57:42.734496+0000","last_clean":"2024-03-16T17:57:42.734496+0000","last_became_active":"2024-03-16T17:57:40.367738+0000","last_became_peered":"2024-03-16T17:57:40.367738+0000","last_unstale":"2024-03-16T17:57:42.734496+0000","last_undegraded":"2024-03-16T17:57:42.734496+0000","last_fullsized":"2024-03-16T17:57:42.734496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732742+0000","last_change":"2024-03-16T17:57:41.378107+0000","last_active":"2024-03-16T17:57:42.732742+0000","last_peered":"2024-03-16T17:57:42.732742+0000","last_clean":"2024-03-16T17:57:42.732742+0000","last_became_active":"2024-03-16T17:57:41.375872+0000","last_became_peered":"2024-03-16T17:57:41.375872+0000","last_unstale":"2024-03-16T17:57:42.732742+0000","last_undegraded":"2024-03-16T17:57:42.732742+0000","last_fullsized":"2024-03-16T17:57:42.732742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733377+0000","last_change":"2024-03-16T17:57:40.375279+0000","last_active":"2024-03-16T17:57:42.733377+0000","last_peered":"2024-03-16T17:57:42.733377+0000","last_clean":"2024-03-16T17:57:42.733377+0000","last_became_active":"2024-03-16T17:57:40.374936+0000","last_became_peered":"2024-03-16T17:57:40.374936+0000","last_unstale":"2024-03-16T17:57:42.733377+0000","last_undegraded":"2024-03-16T17:57:42.733377+0000","last_fullsized":"2024-03-16T17:57:42.733377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:07.628213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733869+0000","last_change":"2024-03-16T17:57:40.372178+0000","last_active":"2024-03-16T17:57:42.733869+0000","last_peered":"2024-03-16T17:57:42.733869+0000","last_clean":"2024-03-16T17:57:42.733869+0000","last_became_active":"2024-03-16T17:57:40.370709+0000","last_became_peered":"2024-03-16T17:57:40.370709+0000","last_unstale":"2024-03-16T17:57:42.733869+0000","last_undegraded":"2024-03-16T17:57:42.733869+0000","last_fullsized":"2024-03-16T17:57:42.733869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:23:36.985797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732131+0000","last_change":"2024-03-16T17:57:41.397372+0000","last_active":"2024-03-16T17:57:42.732131+0000","last_peered":"2024-03-16T17:57:42.732131+0000","last_clean":"2024-03-16T17:57:42.732131+0000","last_became_active":"2024-03-16T17:57:41.397032+0000","last_became_peered":"2024-03-16T17:57:41.397032+0000","last_unstale":"2024-03-16T17:57:42.732131+0000","last_undegraded":"2024-03-16T17:57:42.732131+0000","last_fullsized":"2024-03-16T17:57:42.732131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374381+0000","last_change":"2024-03-16T17:57:40.362691+0000","last_active":"2024-03-16T17:57:42.374381+0000","last_peered":"2024-03-16T17:57:42.374381+0000","last_clean":"2024-03-16T17:57:42.374381+0000","last_became_active":"2024-03-16T17:57:40.361668+0000","last_became_peered":"2024-03-16T17:57:40.361668+0000","last_unstale":"2024-03-16T17:57:42.374381+0000","last_undegraded":"2024-03-16T17:57:42.374381+0000","last_fullsized":"2024-03-16T17:57:42.374381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369126+0000","last_change":"2024-03-16T17:57:41.393663+0000","last_active":"2024-03-16T17:57:42.369126+0000","last_peered":"2024-03-16T17:57:42.369126+0000","last_clean":"2024-03-16T17:57:42.369126+0000","last_became_active":"2024-03-16T17:57:41.393327+0000","last_became_peered":"2024-03-16T17:57:41.393327+0000","last_unstale":"2024-03-16T17:57:42.369126+0000","last_undegraded":"2024-03-16T17:57:42.369126+0000","last_fullsized":"2024-03-16T17:57:42.369126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733784+0000","last_change":"2024-03-16T17:57:40.377710+0000","last_active":"2024-03-16T17:57:42.733784+0000","last_peered":"2024-03-16T17:57:42.733784+0000","last_clean":"2024-03-16T17:57:42.733784+0000","last_became_active":"2024-03-16T17:57:40.377427+0000","last_became_peered":"2024-03-16T17:57:40.377427+0000","last_unstale":"2024-03-16T17:57:42.733784+0000","last_undegraded":"2024-03-16T17:57:42.733784+0000","last_fullsized":"2024-03-16T17:57:42.733784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376320+0000","last_change":"2024-03-16T17:57:41.413272+0000","last_active":"2024-03-16T17:57:42.376320+0000","last_peered":"2024-03-16T17:57:42.376320+0000","last_clean":"2024-03-16T17:57:42.376320+0000","last_became_active":"2024-03-16T17:57:41.412989+0000","last_became_peered":"2024-03-16T17:57:41.412989+0000","last_unstale":"2024-03-16T17:57:42.376320+0000","last_undegraded":"2024-03-16T17:57:42.376320+0000","last_fullsized":"2024-03-16T17:57:42.376320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:36:37.396828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376311+0000","last_change":"2024-03-16T17:57:40.380326+0000","last_active":"2024-03-16T17:57:42.376311+0000","last_peered":"2024-03-16T17:57:42.376311+0000","last_clean":"2024-03-16T17:57:42.376311+0000","last_became_active":"2024-03-16T17:57:40.380003+0000","last_became_peered":"2024-03-16T17:57:40.380003+0000","last_unstale":"2024-03-16T17:57:42.376311+0000","last_undegraded":"2024-03-16T17:57:42.376311+0000","last_fullsized":"2024-03-16T17:57:42.376311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:07.628213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733226+0000","last_change":"2024-03-16T17:57:41.407675+0000","last_active":"2024-03-16T17:57:42.733226+0000","last_peered":"2024-03-16T17:57:42.733226+0000","last_clean":"2024-03-16T17:57:42.733226+0000","last_became_active":"2024-03-16T17:57:41.407412+0000","last_became_peered":"2024-03-16T17:57:41.407412+0000","last_unstale":"2024-03-16T17:57:42.733226+0000","last_undegraded":"2024-03-16T17:57:42.733226+0000","last_fullsized":"2024-03-16T17:57:42.733226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370065+0000","last_change":"2024-03-16T17:57:40.375361+0000","last_active":"2024-03-16T17:57:42.370065+0000","last_peered":"2024-03-16T17:57:42.370065+0000","last_clean":"2024-03-16T17:57:42.370065+0000","last_became_active":"2024-03-16T17:57:40.375086+0000","last_became_peered":"2024-03-16T17:57:40.375086+0000","last_unstale":"2024-03-16T17:57:42.370065+0000","last_undegraded":"2024-03-16T17:57:42.370065+0000","last_fullsized":"2024-03-16T17:57:42.370065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373846+0000","last_change":"2024-03-16T17:57:41.404551+0000","last_active":"2024-03-16T17:57:42.373846+0000","last_peered":"2024-03-16T17:57:42.373846+0000","last_clean":"2024-03-16T17:57:42.373846+0000","last_became_active":"2024-03-16T17:57:41.404266+0000","last_became_peered":"2024-03-16T17:57:41.404266+0000","last_unstale":"2024-03-16T17:57:42.373846+0000","last_undegraded":"2024-03-16T17:57:42.373846+0000","last_fullsized":"2024-03-16T17:57:42.373846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:55:54.818895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733842+0000","last_change":"2024-03-16T17:57:40.376223+0000","last_active":"2024-03-16T17:57:42.733842+0000","last_peered":"2024-03-16T17:57:42.733842+0000","last_clean":"2024-03-16T17:57:42.733842+0000","last_became_active":"2024-03-16T17:57:40.375873+0000","last_became_peered":"2024-03-16T17:57:40.375873+0000","last_unstale":"2024-03-16T17:57:42.733842+0000","last_undegraded":"2024-03-16T17:57:42.733842+0000","last_fullsized":"2024-03-16T17:57:42.733842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735456+0000","last_change":"2024-03-16T17:57:41.403063+0000","last_active":"2024-03-16T17:57:42.735456+0000","last_peered":"2024-03-16T17:57:42.735456+0000","last_clean":"2024-03-16T17:57:42.735456+0000","last_became_active":"2024-03-16T17:57:41.402736+0000","last_became_peered":"2024-03-16T17:57:41.402736+0000","last_unstale":"2024-03-16T17:57:42.735456+0000","last_undegraded":"2024-03-16T17:57:42.735456+0000","last_fullsized":"2024-03-16T17:57:42.735456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:10:39.722613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732825+0000","last_change":"2024-03-16T17:57:40.363660+0000","last_active":"2024-03-16T17:57:42.732825+0000","last_peered":"2024-03-16T17:57:42.732825+0000","last_clean":"2024-03-16T17:57:42.732825+0000","last_became_active":"2024-03-16T17:57:40.363245+0000","last_became_peered":"2024-03-16T17:57:40.363245+0000","last_unstale":"2024-03-16T17:57:42.732825+0000","last_undegraded":"2024-03-16T17:57:42.732825+0000","last_fullsized":"2024-03-16T17:57:42.732825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:45:57.586553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732613+0000","last_change":"2024-03-16T17:57:41.398106+0000","last_active":"2024-03-16T17:57:42.732613+0000","last_peered":"2024-03-16T17:57:42.732613+0000","last_clean":"2024-03-16T17:57:42.732613+0000","last_became_active":"2024-03-16T17:57:41.396446+0000","last_became_peered":"2024-03-16T17:57:41.396446+0000","last_unstale":"2024-03-16T17:57:42.732613+0000","last_undegraded":"2024-03-16T17:57:42.732613+0000","last_fullsized":"2024-03-16T17:57:42.732613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:42:47.316038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375786+0000","last_change":"2024-03-16T17:57:40.377535+0000","last_active":"2024-03-16T17:57:42.375786+0000","last_peered":"2024-03-16T17:57:42.375786+0000","last_clean":"2024-03-16T17:57:42.375786+0000","last_became_active":"2024-03-16T17:57:40.377105+0000","last_became_peered":"2024-03-16T17:57:40.377105+0000","last_unstale":"2024-03-16T17:57:42.375786+0000","last_undegraded":"2024-03-16T17:57:42.375786+0000","last_fullsized":"2024-03-16T17:57:42.375786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735111+0000","last_change":"2024-03-16T17:57:41.407322+0000","last_active":"2024-03-16T17:57:42.735111+0000","last_peered":"2024-03-16T17:57:42.735111+0000","last_clean":"2024-03-16T17:57:42.735111+0000","last_became_active":"2024-03-16T17:57:41.403879+0000","last_became_peered":"2024-03-16T17:57:41.403879+0000","last_unstale":"2024-03-16T17:57:42.735111+0000","last_undegraded":"2024-03-16T17:57:42.735111+0000","last_fullsized":"2024-03-16T17:57:42.735111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:16:17.454713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732837+0000","last_change":"2024-03-16T17:57:40.368238+0000","last_active":"2024-03-16T17:57:42.732837+0000","last_peered":"2024-03-16T17:57:42.732837+0000","last_clean":"2024-03-16T17:57:42.732837+0000","last_became_active":"2024-03-16T17:57:40.367367+0000","last_became_peered":"2024-03-16T17:57:40.367367+0000","last_unstale":"2024-03-16T17:57:42.732837+0000","last_undegraded":"2024-03-16T17:57:42.732837+0000","last_fullsized":"2024-03-16T17:57:42.732837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:26.104046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732994+0000","last_change":"2024-03-16T17:57:41.389244+0000","last_active":"2024-03-16T17:57:42.732994+0000","last_peered":"2024-03-16T17:57:42.732994+0000","last_clean":"2024-03-16T17:57:42.732994+0000","last_became_active":"2024-03-16T17:57:41.388858+0000","last_became_peered":"2024-03-16T17:57:41.388858+0000","last_unstale":"2024-03-16T17:57:42.732994+0000","last_undegraded":"2024-03-16T17:57:42.732994+0000","last_fullsized":"2024-03-16T17:57:42.732994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733311+0000","last_change":"2024-03-16T17:57:40.365096+0000","last_active":"2024-03-16T17:57:42.733311+0000","last_peered":"2024-03-16T17:57:42.733311+0000","last_clean":"2024-03-16T17:57:42.733311+0000","last_became_active":"2024-03-16T17:57:40.364700+0000","last_became_peered":"2024-03-16T17:57:40.364700+0000","last_unstale":"2024-03-16T17:57:42.733311+0000","last_undegraded":"2024-03-16T17:57:42.733311+0000","last_fullsized":"2024-03-16T17:57:42.733311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:53:51.562074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734633+0000","last_change":"2024-03-16T17:57:41.405056+0000","last_active":"2024-03-16T17:57:42.734633+0000","last_peered":"2024-03-16T17:57:42.734633+0000","last_clean":"2024-03-16T17:57:42.734633+0000","last_became_active":"2024-03-16T17:57:41.403286+0000","last_became_peered":"2024-03-16T17:57:41.403286+0000","last_unstale":"2024-03-16T17:57:42.734633+0000","last_undegraded":"2024-03-16T17:57:42.734633+0000","last_fullsized":"2024-03-16T17:57:42.734633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374263+0000","last_change":"2024-03-16T17:57:40.365256+0000","last_active":"2024-03-16T17:57:42.374263+0000","last_peered":"2024-03-16T17:57:42.374263+0000","last_clean":"2024-03-16T17:57:42.374263+0000","last_became_active":"2024-03-16T17:57:40.362214+0000","last_became_peered":"2024-03-16T17:57:40.362214+0000","last_unstale":"2024-03-16T17:57:42.374263+0000","last_undegraded":"2024-03-16T17:57:42.374263+0000","last_fullsized":"2024-03-16T17:57:42.374263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:22.039976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735167+0000","last_change":"2024-03-16T17:57:41.413401+0000","last_active":"2024-03-16T17:57:42.735167+0000","last_peered":"2024-03-16T17:57:42.735167+0000","last_clean":"2024-03-16T17:57:42.735167+0000","last_became_active":"2024-03-16T17:57:41.413163+0000","last_became_peered":"2024-03-16T17:57:41.413163+0000","last_unstale":"2024-03-16T17:57:42.735167+0000","last_undegraded":"2024-03-16T17:57:42.735167+0000","last_fullsized":"2024-03-16T17:57:42.735167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:01:51.467418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734097+0000","last_change":"2024-03-16T17:57:40.370143+0000","last_active":"2024-03-16T17:57:42.734097+0000","last_peered":"2024-03-16T17:57:42.734097+0000","last_clean":"2024-03-16T17:57:42.734097+0000","last_became_active":"2024-03-16T17:57:40.369743+0000","last_became_peered":"2024-03-16T17:57:40.369743+0000","last_unstale":"2024-03-16T17:57:42.734097+0000","last_undegraded":"2024-03-16T17:57:42.734097+0000","last_fullsized":"2024-03-16T17:57:42.734097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:32:24.057779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736551+0000","last_change":"2024-03-16T17:57:41.413116+0000","last_active":"2024-03-16T17:57:42.736551+0000","last_peered":"2024-03-16T17:57:42.736551+0000","last_clean":"2024-03-16T17:57:42.736551+0000","last_became_active":"2024-03-16T17:57:41.412868+0000","last_became_peered":"2024-03-16T17:57:41.412868+0000","last_unstale":"2024-03-16T17:57:42.736551+0000","last_undegraded":"2024-03-16T17:57:42.736551+0000","last_fullsized":"2024-03-16T17:57:42.736551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:20:56.534791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370370+0000","last_change":"2024-03-16T17:57:40.364807+0000","last_active":"2024-03-16T17:57:42.370370+0000","last_peered":"2024-03-16T17:57:42.370370+0000","last_clean":"2024-03-16T17:57:42.370370+0000","last_became_active":"2024-03-16T17:57:40.363383+0000","last_became_peered":"2024-03-16T17:57:40.363383+0000","last_unstale":"2024-03-16T17:57:42.370370+0000","last_undegraded":"2024-03-16T17:57:42.370370+0000","last_fullsized":"2024-03-16T17:57:42.370370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:16.023603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375496+0000","last_change":"2024-03-16T17:57:41.402990+0000","last_active":"2024-03-16T17:57:42.375496+0000","last_peered":"2024-03-16T17:57:42.375496+0000","last_clean":"2024-03-16T17:57:42.375496+0000","last_became_active":"2024-03-16T17:57:41.402696+0000","last_became_peered":"2024-03-16T17:57:41.402696+0000","last_unstale":"2024-03-16T17:57:42.375496+0000","last_undegraded":"2024-03-16T17:57:42.375496+0000","last_fullsized":"2024-03-16T17:57:42.375496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:16:17.454713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733543+0000","last_change":"2024-03-16T17:57:40.378607+0000","last_active":"2024-03-16T17:57:42.733543+0000","last_peered":"2024-03-16T17:57:42.733543+0000","last_clean":"2024-03-16T17:57:42.733543+0000","last_became_active":"2024-03-16T17:57:40.378291+0000","last_became_peered":"2024-03-16T17:57:40.378291+0000","last_unstale":"2024-03-16T17:57:42.733543+0000","last_undegraded":"2024-03-16T17:57:42.733543+0000","last_fullsized":"2024-03-16T17:57:42.733543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:16.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735929+0000","last_change":"2024-03-16T17:57:41.409430+0000","last_active":"2024-03-16T17:57:42.735929+0000","last_peered":"2024-03-16T17:57:42.735929+0000","last_clean":"2024-03-16T17:57:42.735929+0000","last_became_active":"2024-03-16T17:57:41.409209+0000","last_became_peered":"2024-03-16T17:57:41.409209+0000","last_unstale":"2024-03-16T17:57:42.735929+0000","last_undegraded":"2024-03-16T17:57:42.735929+0000","last_fullsized":"2024-03-16T17:57:42.735929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:58.061341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733228+0000","last_change":"2024-03-16T17:57:35.560172+0000","last_active":"2024-03-16T17:57:42.733228+0000","last_peered":"2024-03-16T17:57:42.733228+0000","last_clean":"2024-03-16T17:57:42.733228+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T17:57:42.733228+0000","last_undegraded":"2024-03-16T17:57:42.733228+0000","last_fullsized":"2024-03-16T17:57:42.733228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_clean_scrub_stamp":"2024-03-16T17:57:34.174437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:20:03.039404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374626+0000","last_change":"2024-03-16T17:57:40.372426+0000","last_active":"2024-03-16T17:57:42.374626+0000","last_peered":"2024-03-16T17:57:42.374626+0000","last_clean":"2024-03-16T17:57:42.374626+0000","last_became_active":"2024-03-16T17:57:40.372097+0000","last_became_peered":"2024-03-16T17:57:40.372097+0000","last_unstale":"2024-03-16T17:57:42.374626+0000","last_undegraded":"2024-03-16T17:57:42.374626+0000","last_fullsized":"2024-03-16T17:57:42.374626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734031+0000","last_change":"2024-03-16T17:57:41.401095+0000","last_active":"2024-03-16T17:57:42.734031+0000","last_peered":"2024-03-16T17:57:42.734031+0000","last_clean":"2024-03-16T17:57:42.734031+0000","last_became_active":"2024-03-16T17:57:41.400788+0000","last_became_peered":"2024-03-16T17:57:41.400788+0000","last_unstale":"2024-03-16T17:57:42.734031+0000","last_undegraded":"2024-03-16T17:57:42.734031+0000","last_fullsized":"2024-03-16T17:57:42.734031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733259+0000","last_change":"2024-03-16T17:57:40.369748+0000","last_active":"2024-03-16T17:57:42.733259+0000","last_peered":"2024-03-16T17:57:42.733259+0000","last_clean":"2024-03-16T17:57:42.733259+0000","last_became_active":"2024-03-16T17:57:40.368850+0000","last_became_peered":"2024-03-16T17:57:40.368850+0000","last_unstale":"2024-03-16T17:57:42.733259+0000","last_undegraded":"2024-03-16T17:57:42.733259+0000","last_fullsized":"2024-03-16T17:57:42.733259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:31:16.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732283+0000","last_change":"2024-03-16T17:57:41.373872+0000","last_active":"2024-03-16T17:57:42.732283+0000","last_peered":"2024-03-16T17:57:42.732283+0000","last_clean":"2024-03-16T17:57:42.732283+0000","last_became_active":"2024-03-16T17:57:41.373478+0000","last_became_peered":"2024-03-16T17:57:41.373478+0000","last_unstale":"2024-03-16T17:57:42.732283+0000","last_undegraded":"2024-03-16T17:57:42.732283+0000","last_fullsized":"2024-03-16T17:57:42.732283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:04:23.189434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733503+0000","last_change":"2024-03-16T17:57:40.373316+0000","last_active":"2024-03-16T17:57:42.733503+0000","last_peered":"2024-03-16T17:57:42.733503+0000","last_clean":"2024-03-16T17:57:42.733503+0000","last_became_active":"2024-03-16T17:57:40.373015+0000","last_became_peered":"2024-03-16T17:57:40.373015+0000","last_unstale":"2024-03-16T17:57:42.733503+0000","last_undegraded":"2024-03-16T17:57:42.733503+0000","last_fullsized":"2024-03-16T17:57:42.733503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:42:38.164765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369425+0000","last_change":"2024-03-16T17:57:41.383198+0000","last_active":"2024-03-16T17:57:42.369425+0000","last_peered":"2024-03-16T17:57:42.369425+0000","last_clean":"2024-03-16T17:57:42.369425+0000","last_became_active":"2024-03-16T17:57:41.382886+0000","last_became_peered":"2024-03-16T17:57:41.382886+0000","last_unstale":"2024-03-16T17:57:42.369425+0000","last_undegraded":"2024-03-16T17:57:42.369425+0000","last_fullsized":"2024-03-16T17:57:42.369425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375951+0000","last_change":"2024-03-16T17:57:40.368934+0000","last_active":"2024-03-16T17:57:42.375951+0000","last_peered":"2024-03-16T17:57:42.375951+0000","last_clean":"2024-03-16T17:57:42.375951+0000","last_became_active":"2024-03-16T17:57:40.368496+0000","last_became_peered":"2024-03-16T17:57:40.368496+0000","last_unstale":"2024-03-16T17:57:42.375951+0000","last_undegraded":"2024-03-16T17:57:42.375951+0000","last_fullsized":"2024-03-16T17:57:42.375951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373158+0000","last_change":"2024-03-16T17:57:41.377235+0000","last_active":"2024-03-16T17:57:42.373158+0000","last_peered":"2024-03-16T17:57:42.373158+0000","last_clean":"2024-03-16T17:57:42.373158+0000","last_became_active":"2024-03-16T17:57:41.376796+0000","last_became_peered":"2024-03-16T17:57:41.376796+0000","last_unstale":"2024-03-16T17:57:42.373158+0000","last_undegraded":"2024-03-16T17:57:42.373158+0000","last_fullsized":"2024-03-16T17:57:42.373158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:24:23.606350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369894+0000","last_change":"2024-03-16T17:57:40.365747+0000","last_active":"2024-03-16T17:57:42.369894+0000","last_peered":"2024-03-16T17:57:42.369894+0000","last_clean":"2024-03-16T17:57:42.369894+0000","last_became_active":"2024-03-16T17:57:40.364579+0000","last_became_peered":"2024-03-16T17:57:40.364579+0000","last_unstale":"2024-03-16T17:57:42.369894+0000","last_undegraded":"2024-03-16T17:57:42.369894+0000","last_fullsized":"2024-03-16T17:57:42.369894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376270+0000","last_change":"2024-03-16T17:57:41.404470+0000","last_active":"2024-03-16T17:57:42.376270+0000","last_peered":"2024-03-16T17:57:42.376270+0000","last_clean":"2024-03-16T17:57:42.376270+0000","last_became_active":"2024-03-16T17:57:41.404100+0000","last_became_peered":"2024-03-16T17:57:41.404100+0000","last_unstale":"2024-03-16T17:57:42.376270+0000","last_undegraded":"2024-03-16T17:57:42.376270+0000","last_fullsized":"2024-03-16T17:57:42.376270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:50:09.742348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736227+0000","last_change":"2024-03-16T17:57:40.370118+0000","last_active":"2024-03-16T17:57:42.736227+0000","last_peered":"2024-03-16T17:57:42.736227+0000","last_clean":"2024-03-16T17:57:42.736227+0000","last_became_active":"2024-03-16T17:57:40.369751+0000","last_became_peered":"2024-03-16T17:57:40.369751+0000","last_unstale":"2024-03-16T17:57:42.736227+0000","last_undegraded":"2024-03-16T17:57:42.736227+0000","last_fullsized":"2024-03-16T17:57:42.736227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:04:22.171852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369793+0000","last_change":"2024-03-16T17:57:41.384659+0000","last_active":"2024-03-16T17:57:42.369793+0000","last_peered":"2024-03-16T17:57:42.369793+0000","last_clean":"2024-03-16T17:57:42.369793+0000","last_became_active":"2024-03-16T17:57:41.384374+0000","last_became_peered":"2024-03-16T17:57:41.384374+0000","last_unstale":"2024-03-16T17:57:42.369793+0000","last_undegraded":"2024-03-16T17:57:42.369793+0000","last_fullsized":"2024-03-16T17:57:42.369793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:43.209075+0000","last_change":"2024-03-16T17:57:40.377982+0000","last_active":"2024-03-16T17:57:43.209075+0000","last_peered":"2024-03-16T17:57:43.209075+0000","last_clean":"2024-03-16T17:57:43.209075+0000","last_became_active":"2024-03-16T17:57:40.377669+0000","last_became_peered":"2024-03-16T17:57:40.377669+0000","last_unstale":"2024-03-16T17:57:43.209075+0000","last_undegraded":"2024-03-16T17:57:43.209075+0000","last_fullsized":"2024-03-16T17:57:43.209075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:31:16.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735021+0000","last_change":"2024-03-16T17:57:41.400625+0000","last_active":"2024-03-16T17:57:42.735021+0000","last_peered":"2024-03-16T17:57:42.735021+0000","last_clean":"2024-03-16T17:57:42.735021+0000","last_became_active":"2024-03-16T17:57:41.398361+0000","last_became_peered":"2024-03-16T17:57:41.398361+0000","last_unstale":"2024-03-16T17:57:42.735021+0000","last_undegraded":"2024-03-16T17:57:42.735021+0000","last_fullsized":"2024-03-16T17:57:42.735021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733837+0000","last_change":"2024-03-16T17:57:40.379426+0000","last_active":"2024-03-16T17:57:42.733837+0000","last_peered":"2024-03-16T17:57:42.733837+0000","last_clean":"2024-03-16T17:57:42.733837+0000","last_became_active":"2024-03-16T17:57:40.379136+0000","last_became_peered":"2024-03-16T17:57:40.379136+0000","last_unstale":"2024-03-16T17:57:42.733837+0000","last_undegraded":"2024-03-16T17:57:42.733837+0000","last_fullsized":"2024-03-16T17:57:42.733837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732957+0000","last_change":"2024-03-16T17:57:41.379286+0000","last_active":"2024-03-16T17:57:42.732957+0000","last_peered":"2024-03-16T17:57:42.732957+0000","last_clean":"2024-03-16T17:57:42.732957+0000","last_became_active":"2024-03-16T17:57:41.378998+0000","last_became_peered":"2024-03-16T17:57:41.378998+0000","last_unstale":"2024-03-16T17:57:42.732957+0000","last_undegraded":"2024-03-16T17:57:42.732957+0000","last_fullsized":"2024-03-16T17:57:42.732957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732976+0000","last_change":"2024-03-16T17:57:40.369141+0000","last_active":"2024-03-16T17:57:42.732976+0000","last_peered":"2024-03-16T17:57:42.732976+0000","last_clean":"2024-03-16T17:57:42.732976+0000","last_became_active":"2024-03-16T17:57:40.368542+0000","last_became_peered":"2024-03-16T17:57:40.368542+0000","last_unstale":"2024-03-16T17:57:42.732976+0000","last_undegraded":"2024-03-16T17:57:42.732976+0000","last_fullsized":"2024-03-16T17:57:42.732976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.372617+0000","last_change":"2024-03-16T17:57:41.375830+0000","last_active":"2024-03-16T17:57:42.372617+0000","last_peered":"2024-03-16T17:57:42.372617+0000","last_clean":"2024-03-16T17:57:42.372617+0000","last_became_active":"2024-03-16T17:57:41.375515+0000","last_became_peered":"2024-03-16T17:57:41.375515+0000","last_unstale":"2024-03-16T17:57:42.372617+0000","last_undegraded":"2024-03-16T17:57:42.372617+0000","last_fullsized":"2024-03-16T17:57:42.372617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:32:25.075362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369244+0000","last_change":"2024-03-16T17:57:40.367142+0000","last_active":"2024-03-16T17:57:42.369244+0000","last_peered":"2024-03-16T17:57:42.369244+0000","last_clean":"2024-03-16T17:57:42.369244+0000","last_became_active":"2024-03-16T17:57:40.366778+0000","last_became_peered":"2024-03-16T17:57:40.366778+0000","last_unstale":"2024-03-16T17:57:42.369244+0000","last_undegraded":"2024-03-16T17:57:42.369244+0000","last_fullsized":"2024-03-16T17:57:42.369244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732297+0000","last_change":"2024-03-16T17:57:41.388886+0000","last_active":"2024-03-16T17:57:42.732297+0000","last_peered":"2024-03-16T17:57:42.732297+0000","last_clean":"2024-03-16T17:57:42.732297+0000","last_became_active":"2024-03-16T17:57:41.388588+0000","last_became_peered":"2024-03-16T17:57:41.388588+0000","last_unstale":"2024-03-16T17:57:42.732297+0000","last_undegraded":"2024-03-16T17:57:42.732297+0000","last_fullsized":"2024-03-16T17:57:42.732297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369599+0000","last_change":"2024-03-16T17:57:40.371549+0000","last_active":"2024-03-16T17:57:42.369599+0000","last_peered":"2024-03-16T17:57:42.369599+0000","last_clean":"2024-03-16T17:57:42.369599+0000","last_became_active":"2024-03-16T17:57:40.371257+0000","last_became_peered":"2024-03-16T17:57:40.371257+0000","last_unstale":"2024-03-16T17:57:42.369599+0000","last_undegraded":"2024-03-16T17:57:42.369599+0000","last_fullsized":"2024-03-16T17:57:42.369599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734385+0000","last_change":"2024-03-16T17:57:41.392961+0000","last_active":"2024-03-16T17:57:42.734385+0000","last_peered":"2024-03-16T17:57:42.734385+0000","last_clean":"2024-03-16T17:57:42.734385+0000","last_became_active":"2024-03-16T17:57:41.392665+0000","last_became_peered":"2024-03-16T17:57:41.392665+0000","last_unstale":"2024-03-16T17:57:42.734385+0000","last_undegraded":"2024-03-16T17:57:42.734385+0000","last_fullsized":"2024-03-16T17:57:42.734385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735443+0000","last_change":"2024-03-16T17:57:40.376135+0000","last_active":"2024-03-16T17:57:42.735443+0000","last_peered":"2024-03-16T17:57:42.735443+0000","last_clean":"2024-03-16T17:57:42.735443+0000","last_became_active":"2024-03-16T17:57:40.375292+0000","last_became_peered":"2024-03-16T17:57:40.375292+0000","last_unstale":"2024-03-16T17:57:42.735443+0000","last_undegraded":"2024-03-16T17:57:42.735443+0000","last_fullsized":"2024-03-16T17:57:42.735443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:16.023603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733288+0000","last_change":"2024-03-16T17:57:41.394268+0000","last_active":"2024-03-16T17:57:42.733288+0000","last_peered":"2024-03-16T17:57:42.733288+0000","last_clean":"2024-03-16T17:57:42.733288+0000","last_became_active":"2024-03-16T17:57:41.393941+0000","last_became_peered":"2024-03-16T17:57:41.393941+0000","last_unstale":"2024-03-16T17:57:42.733288+0000","last_undegraded":"2024-03-16T17:57:42.733288+0000","last_fullsized":"2024-03-16T17:57:42.733288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376119+0000","last_change":"2024-03-16T17:57:41.402175+0000","last_active":"2024-03-16T17:57:42.376119+0000","last_peered":"2024-03-16T17:57:42.376119+0000","last_clean":"2024-03-16T17:57:42.376119+0000","last_became_active":"2024-03-16T17:57:41.401851+0000","last_became_peered":"2024-03-16T17:57:41.401851+0000","last_unstale":"2024-03-16T17:57:42.376119+0000","last_undegraded":"2024-03-16T17:57:42.376119+0000","last_fullsized":"2024-03-16T17:57:42.376119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:58.061341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734977+0000","last_change":"2024-03-16T17:57:40.377332+0000","last_active":"2024-03-16T17:57:42.734977+0000","last_peered":"2024-03-16T17:57:42.734977+0000","last_clean":"2024-03-16T17:57:42.734977+0000","last_became_active":"2024-03-16T17:57:40.377028+0000","last_became_peered":"2024-03-16T17:57:40.377028+0000","last_unstale":"2024-03-16T17:57:42.734977+0000","last_undegraded":"2024-03-16T17:57:42.734977+0000","last_fullsized":"2024-03-16T17:57:42.734977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375244+0000","last_change":"2024-03-16T17:57:41.402689+0000","last_active":"2024-03-16T17:57:42.375244+0000","last_peered":"2024-03-16T17:57:42.375244+0000","last_clean":"2024-03-16T17:57:42.375244+0000","last_became_active":"2024-03-16T17:57:41.401253+0000","last_became_peered":"2024-03-16T17:57:41.401253+0000","last_unstale":"2024-03-16T17:57:42.375244+0000","last_undegraded":"2024-03-16T17:57:42.375244+0000","last_fullsized":"2024-03-16T17:57:42.375244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735773+0000","last_change":"2024-03-16T17:57:40.379396+0000","last_active":"2024-03-16T17:57:42.735773+0000","last_peered":"2024-03-16T17:57:42.735773+0000","last_clean":"2024-03-16T17:57:42.735773+0000","last_became_active":"2024-03-16T17:57:40.379058+0000","last_became_peered":"2024-03-16T17:57:40.379058+0000","last_unstale":"2024-03-16T17:57:42.735773+0000","last_undegraded":"2024-03-16T17:57:42.735773+0000","last_fullsized":"2024-03-16T17:57:42.735773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:53:59.451003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732999+0000","last_change":"2024-03-16T17:57:41.407792+0000","last_active":"2024-03-16T17:57:42.732999+0000","last_peered":"2024-03-16T17:57:42.732999+0000","last_clean":"2024-03-16T17:57:42.732999+0000","last_became_active":"2024-03-16T17:57:41.407412+0000","last_became_peered":"2024-03-16T17:57:41.407412+0000","last_unstale":"2024-03-16T17:57:42.732999+0000","last_undegraded":"2024-03-16T17:57:42.732999+0000","last_fullsized":"2024-03-16T17:57:42.732999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735117+0000","last_change":"2024-03-16T17:57:40.366569+0000","last_active":"2024-03-16T17:57:42.735117+0000","last_peered":"2024-03-16T17:57:42.735117+0000","last_clean":"2024-03-16T17:57:42.735117+0000","last_became_active":"2024-03-16T17:57:40.362328+0000","last_became_peered":"2024-03-16T17:57:40.362328+0000","last_unstale":"2024-03-16T17:57:42.735117+0000","last_undegraded":"2024-03-16T17:57:42.735117+0000","last_fullsized":"2024-03-16T17:57:42.735117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734985+0000","last_change":"2024-03-16T17:57:41.400212+0000","last_active":"2024-03-16T17:57:42.734985+0000","last_peered":"2024-03-16T17:57:42.734985+0000","last_clean":"2024-03-16T17:57:42.734985+0000","last_became_active":"2024-03-16T17:57:41.399937+0000","last_became_peered":"2024-03-16T17:57:41.399937+0000","last_unstale":"2024-03-16T17:57:42.734985+0000","last_undegraded":"2024-03-16T17:57:42.734985+0000","last_fullsized":"2024-03-16T17:57:42.734985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733309+0000","last_change":"2024-03-16T17:57:40.368635+0000","last_active":"2024-03-16T17:57:42.733309+0000","last_peered":"2024-03-16T17:57:42.733309+0000","last_clean":"2024-03-16T17:57:42.733309+0000","last_became_active":"2024-03-16T17:57:40.368370+0000","last_became_peered":"2024-03-16T17:57:40.368370+0000","last_unstale":"2024-03-16T17:57:42.733309+0000","last_undegraded":"2024-03-16T17:57:42.733309+0000","last_fullsized":"2024-03-16T17:57:42.733309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:19.918231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":665113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":74253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T17:57:51.290 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T17:57:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.528+0000 7f6141e00640 1 Processor -- start 2024-03-16T17:57:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.528+0000 7f6141e00640 1 -- start start 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.529+0000 7f6141e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c144400 0x7f613c0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.529+0000 7f6141e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f613c15ba20 0x7f613c0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.529+0000 7f6141e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613c15e820 con 0x7f613c15f550 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.529+0000 7f6141e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613c15f130 con 0x7f613c15ba20 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.529+0000 7f6141e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c15be90 con 0x7f613c144400 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.530+0000 7f613b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f613c15ba20 0x7f613c0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.530+0000 7f613b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f613c15ba20 0x7f613c0cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35506/0 (socket says 172.21.15.82:35506) 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.530+0000 7f613b400640 1 -- 172.21.15.82:0/3515019830 learned_addr learned my addr 172.21.15.82:0/3515019830 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.530+0000 7f613aa00640 1 --2- 172.21.15.82:0/3515019830 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c144400 0x7f613c0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.530+0000 7f613b400640 1 -- 172.21.15.82:0/3515019830 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c144400 msgr2=0x7f613c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:51.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.530+0000 7f613b400640 1 --2- 172.21.15.82:0/3515019830 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c144400 0x7f613c0cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.530+0000 7f613b400640 1 -- 172.21.15.82:0/3515019830 >> v1:172.21.15.90:6789/0 conn(0x7f613c15f550 legacy=0x7f613c15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:51.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.530+0000 7f613b400640 1 -- 172.21.15.82:0/3515019830 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613c15c080 con 0x7f613c15ba20 2024-03-16T17:57:51.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.530+0000 7f613aa00640 1 --2- 172.21.15.82:0/3515019830 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c144400 0x7f613c0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.531+0000 7f613b400640 1 --2- 172.21.15.82:0/3515019830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 0x7f613c0cfed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6134009e50 tx=0x7f613402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb730a22ad73f6a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f613a000640 1 -- 172.21.15.82:0/3515019830 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613402c9e0 con 0x7f613c15ba20 2024-03-16T17:57:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f613a000640 1 -- 172.21.15.82:0/3515019830 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613402cb40 con 0x7f613c15ba20 2024-03-16T17:57:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f613a000640 1 -- 172.21.15.82:0/3515019830 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61340343f0 con 0x7f613c15ba20 2024-03-16T17:57:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f6141e00640 1 -- 172.21.15.82:0/3515019830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 msgr2=0x7f613c0cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f6141e00640 1 --2- 172.21.15.82:0/3515019830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 0x7f613c0cfed0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6134009e50 tx=0x7f613402be40 comp rx=0 tx=0).stop 2024-03-16T17:57:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f6141e00640 1 -- 172.21.15.82:0/3515019830 shutdown_connections 2024-03-16T17:57:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f6141e00640 1 --2- 172.21.15.82:0/3515019830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 0x7f613c0cfed0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f6141e00640 1 --2- 172.21.15.82:0/3515019830 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c144400 0x7f613c0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f6141e00640 1 -- 172.21.15.82:0/3515019830 >> 172.21.15.82:0/3515019830 conn(0x7f613c06ae70 msgr2=0x7f613c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f6141e00640 1 -- 172.21.15.82:0/3515019830 shutdown_connections 2024-03-16T17:57:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.532+0000 7f6141e00640 1 -- 172.21.15.82:0/3515019830 wait complete. 2024-03-16T17:57:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.534+0000 7f6141e00640 1 Processor -- start 2024-03-16T17:57:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.534+0000 7f6141e00640 1 -- start start 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.535+0000 7f6141e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f613c144400 0x7f613c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.535+0000 7f6141e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 0x7f613c140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.535+0000 7f6141e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c15f550 0x7f613c071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.535+0000 7f6141e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c1416b0 con 0x7f613c15ba20 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.535+0000 7f6141e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c140aa0 con 0x7f613c144400 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.535+0000 7f6141e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613c140d80 con 0x7f613c15f550 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.535+0000 7f6132000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 0x7f613c140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.535+0000 7f6132000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 0x7f613c140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35518/0 (socket says 172.21.15.82:35518) 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.535+0000 7f6132000640 1 -- 172.21.15.82:0/3940634836 learned_addr learned my addr 172.21.15.82:0/3940634836 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f613aa00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f613c144400 0x7f613c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6132000640 1 -- 172.21.15.82:0/3940634836 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c15f550 msgr2=0x7f613c071390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f613b400640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c15f550 0x7f613c071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6132000640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c15f550 0x7f613c071390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6132000640 1 -- 172.21.15.82:0/3940634836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f613c144400 msgr2=0x7f613c1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6132000640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f613c144400 0x7f613c1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6132000640 1 -- 172.21.15.82:0/3940634836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6134009b30 con 0x7f613c15ba20 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f613aa00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f613c144400 0x7f613c1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6132000640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 0x7f613c140130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f612400c430 tx=0x7f612400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6133e00640 1 -- 172.21.15.82:0/3940634836 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124011790 con 0x7f613c15ba20 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6133e00640 1 -- 172.21.15.82:0/3940634836 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61240118f0 con 0x7f613c15ba20 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6133e00640 1 -- 172.21.15.82:0/3940634836 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124011ce0 con 0x7f613c15ba20 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6141e00640 1 -- 172.21.15.82:0/3940634836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c141060 con 0x7f613c15ba20 2024-03-16T17:57:51.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.536+0000 7f6141e00640 1 -- 172.21.15.82:0/3940634836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c140670 con 0x7f613c15ba20 2024-03-16T17:57:51.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.538+0000 7f6141e00640 1 -- 172.21.15.82:0/3940634836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c067f30 con 0x7f613c15ba20 2024-03-16T17:57:51.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.538+0000 7f6133e00640 1 -- 172.21.15.82:0/3940634836 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f612401c460 con 0x7f613c15ba20 2024-03-16T17:57:51.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.539+0000 7f6133e00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f611806b3b0 0x7f611806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:51.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.539+0000 7f6133e00640 1 -- 172.21.15.82:0/3940634836 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6124089af0 con 0x7f613c15ba20 2024-03-16T17:57:51.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.539+0000 7f613aa00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f611806b3b0 0x7f611806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:51.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.540+0000 7f613aa00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f611806b3b0 0x7f611806d870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f612c006fd0 tx=0x7f612c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:51.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.543+0000 7f6133e00640 1 -- 172.21.15.82:0/3940634836 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f61240547e0 con 0x7f613c15ba20 2024-03-16T17:57:51.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.721+0000 7f6141e00640 1 -- 172.21.15.82:0/3940634836 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f613c140890 con 0x7f611806b3b0 2024-03-16T17:57:51.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.729+0000 7f6133e00640 1 -- 172.21.15.82:0/3940634836 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251838 (secure 0 0 0) 0x7f613c140890 con 0x7f611806b3b0 2024-03-16T17:57:51.736 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:51.737 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 -- 172.21.15.82:0/3940634836 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f611806b3b0 msgr2=0x7f611806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f611806b3b0 0x7f611806d870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f612c006fd0 tx=0x7f612c008040 comp rx=0 tx=0).stop 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 -- 172.21.15.82:0/3940634836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 msgr2=0x7f613c140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 0x7f613c140130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f612400c430 tx=0x7f612400c8f0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 -- 172.21.15.82:0/3940634836 shutdown_connections 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f611806b3b0 0x7f611806d870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f613c15f550 0x7f613c071390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f613c15ba20 0x7f613c140130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 --2- 172.21.15.82:0/3940634836 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f613c144400 0x7f613c1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 -- 172.21.15.82:0/3940634836 >> 172.21.15.82:0/3940634836 conn(0x7f613c06ae70 msgr2=0x7f613c15b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 -- 172.21.15.82:0/3940634836 shutdown_connections 2024-03-16T17:57:51.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.734+0000 7f6141e00640 1 -- 172.21.15.82:0/3940634836 wait complete. 2024-03-16T17:57:51.755 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-16T17:57:50.013336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2957312,"data_stored":1778860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.651215"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736016+0000","last_change":"2024-03-16T17:57:41.412158+0000","last_active":"2024-03-16T17:57:42.736016+0000","last_peered":"2024-03-16T17:57:42.736016+0000","last_clean":"2024-03-16T17:57:42.736016+0000","last_became_active":"2024-03-16T17:57:41.411766+0000","last_became_peered":"2024-03-16T17:57:41.411766+0000","last_unstale":"2024-03-16T17:57:42.736016+0000","last_undegraded":"2024-03-16T17:57:42.736016+0000","last_fullsized":"2024-03-16T17:57:42.736016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:50:09.742348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370170+0000","last_change":"2024-03-16T17:57:41.383449+0000","last_active":"2024-03-16T17:57:42.370170+0000","last_peered":"2024-03-16T17:57:42.370170+0000","last_clean":"2024-03-16T17:57:42.370170+0000","last_became_active":"2024-03-16T17:57:41.382165+0000","last_became_peered":"2024-03-16T17:57:41.382165+0000","last_unstale":"2024-03-16T17:57:42.370170+0000","last_undegraded":"2024-03-16T17:57:42.370170+0000","last_fullsized":"2024-03-16T17:57:42.370170+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732651+0000","last_change":"2024-03-16T17:57:41.380082+0000","last_active":"2024-03-16T17:57:42.732651+0000","last_peered":"2024-03-16T17:57:42.732651+0000","last_clean":"2024-03-16T17:57:42.732651+0000","last_became_active":"2024-03-16T17:57:41.379508+0000","last_became_peered":"2024-03-16T17:57:41.379508+0000","last_unstale":"2024-03-16T17:57:42.732651+0000","last_undegraded":"2024-03-16T17:57:42.732651+0000","last_fullsized":"2024-03-16T17:57:42.732651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375083+0000","last_change":"2024-03-16T17:57:41.404364+0000","last_active":"2024-03-16T17:57:42.375083+0000","last_peered":"2024-03-16T17:57:42.375083+0000","last_clean":"2024-03-16T17:57:42.375083+0000","last_became_active":"2024-03-16T17:57:41.404048+0000","last_became_peered":"2024-03-16T17:57:41.404048+0000","last_unstale":"2024-03-16T17:57:42.375083+0000","last_undegraded":"2024-03-16T17:57:42.375083+0000","last_fullsized":"2024-03-16T17:57:42.375083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734821+0000","last_change":"2024-03-16T17:57:41.392327+0000","last_active":"2024-03-16T17:57:42.734821+0000","last_peered":"2024-03-16T17:57:42.734821+0000","last_clean":"2024-03-16T17:57:42.734821+0000","last_became_active":"2024-03-16T17:57:41.392007+0000","last_became_peered":"2024-03-16T17:57:41.392007+0000","last_unstale":"2024-03-16T17:57:42.734821+0000","last_undegraded":"2024-03-16T17:57:42.734821+0000","last_fullsized":"2024-03-16T17:57:42.734821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:48:10.857666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375324+0000","last_change":"2024-03-16T17:57:41.398446+0000","last_active":"2024-03-16T17:57:42.375324+0000","last_peered":"2024-03-16T17:57:42.375324+0000","last_clean":"2024-03-16T17:57:42.375324+0000","last_became_active":"2024-03-16T17:57:41.398135+0000","last_became_peered":"2024-03-16T17:57:41.398135+0000","last_unstale":"2024-03-16T17:57:42.375324+0000","last_undegraded":"2024-03-16T17:57:42.375324+0000","last_fullsized":"2024-03-16T17:57:42.375324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:48:10.857666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734440+0000","last_change":"2024-03-16T17:57:41.405535+0000","last_active":"2024-03-16T17:57:42.734440+0000","last_peered":"2024-03-16T17:57:42.734440+0000","last_clean":"2024-03-16T17:57:42.734440+0000","last_became_active":"2024-03-16T17:57:41.405323+0000","last_became_peered":"2024-03-16T17:57:41.405323+0000","last_unstale":"2024-03-16T17:57:42.734440+0000","last_undegraded":"2024-03-16T17:57:42.734440+0000","last_fullsized":"2024-03-16T17:57:42.734440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.731940+0000","last_change":"2024-03-16T17:57:41.402459+0000","last_active":"2024-03-16T17:57:42.731940+0000","last_peered":"2024-03-16T17:57:42.731940+0000","last_clean":"2024-03-16T17:57:42.731940+0000","last_became_active":"2024-03-16T17:57:41.402103+0000","last_became_peered":"2024-03-16T17:57:41.402103+0000","last_unstale":"2024-03-16T17:57:42.731940+0000","last_undegraded":"2024-03-16T17:57:42.731940+0000","last_fullsized":"2024-03-16T17:57:42.731940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:20.628509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736292+0000","last_change":"2024-03-16T17:57:41.409011+0000","last_active":"2024-03-16T17:57:42.736292+0000","last_peered":"2024-03-16T17:57:42.736292+0000","last_clean":"2024-03-16T17:57:42.736292+0000","last_became_active":"2024-03-16T17:57:41.408811+0000","last_became_peered":"2024-03-16T17:57:41.408811+0000","last_unstale":"2024-03-16T17:57:42.736292+0000","last_undegraded":"2024-03-16T17:57:42.736292+0000","last_fullsized":"2024-03-16T17:57:42.736292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:11:48.511692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734591+0000","last_change":"2024-03-16T17:57:41.401707+0000","last_active":"2024-03-16T17:57:42.734591+0000","last_peered":"2024-03-16T17:57:42.734591+0000","last_clean":"2024-03-16T17:57:42.734591+0000","last_became_active":"2024-03-16T17:57:41.401330+0000","last_became_peered":"2024-03-16T17:57:41.401330+0000","last_unstale":"2024-03-16T17:57:42.734591+0000","last_undegraded":"2024-03-16T17:57:42.734591+0000","last_fullsized":"2024-03-16T17:57:42.734591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733619+0000","last_change":"2024-03-16T17:57:41.399956+0000","last_active":"2024-03-16T17:57:42.733619+0000","last_peered":"2024-03-16T17:57:42.733619+0000","last_clean":"2024-03-16T17:57:42.733619+0000","last_became_active":"2024-03-16T17:57:41.399696+0000","last_became_peered":"2024-03-16T17:57:41.399696+0000","last_unstale":"2024-03-16T17:57:42.733619+0000","last_undegraded":"2024-03-16T17:57:42.733619+0000","last_fullsized":"2024-03-16T17:57:42.733619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373739+0000","last_change":"2024-03-16T17:57:41.397224+0000","last_active":"2024-03-16T17:57:42.373739+0000","last_peered":"2024-03-16T17:57:42.373739+0000","last_clean":"2024-03-16T17:57:42.373739+0000","last_became_active":"2024-03-16T17:57:41.396857+0000","last_became_peered":"2024-03-16T17:57:41.396857+0000","last_unstale":"2024-03-16T17:57:42.373739+0000","last_undegraded":"2024-03-16T17:57:42.373739+0000","last_fullsized":"2024-03-16T17:57:42.373739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:32:52.837630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734251+0000","last_change":"2024-03-16T17:57:41.399774+0000","last_active":"2024-03-16T17:57:42.734251+0000","last_peered":"2024-03-16T17:57:42.734251+0000","last_clean":"2024-03-16T17:57:42.734251+0000","last_became_active":"2024-03-16T17:57:41.399505+0000","last_became_peered":"2024-03-16T17:57:41.399505+0000","last_unstale":"2024-03-16T17:57:42.734251+0000","last_undegraded":"2024-03-16T17:57:42.734251+0000","last_fullsized":"2024-03-16T17:57:42.734251+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732264+0000","last_change":"2024-03-16T17:57:41.380132+0000","last_active":"2024-03-16T17:57:42.732264+0000","last_peered":"2024-03-16T17:57:42.732264+0000","last_clean":"2024-03-16T17:57:42.732264+0000","last_became_active":"2024-03-16T17:57:41.379536+0000","last_became_peered":"2024-03-16T17:57:41.379536+0000","last_unstale":"2024-03-16T17:57:42.732264+0000","last_undegraded":"2024-03-16T17:57:42.732264+0000","last_fullsized":"2024-03-16T17:57:42.732264+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:54:00.468586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369395+0000","last_change":"2024-03-16T17:57:41.393322+0000","last_active":"2024-03-16T17:57:42.369395+0000","last_peered":"2024-03-16T17:57:42.369395+0000","last_clean":"2024-03-16T17:57:42.369395+0000","last_became_active":"2024-03-16T17:57:41.391272+0000","last_became_peered":"2024-03-16T17:57:41.391272+0000","last_unstale":"2024-03-16T17:57:42.369395+0000","last_undegraded":"2024-03-16T17:57:42.369395+0000","last_fullsized":"2024-03-16T17:57:42.369395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374104+0000","last_change":"2024-03-16T17:57:41.399992+0000","last_active":"2024-03-16T17:57:42.374104+0000","last_peered":"2024-03-16T17:57:42.374104+0000","last_clean":"2024-03-16T17:57:42.374104+0000","last_became_active":"2024-03-16T17:57:41.399406+0000","last_became_peered":"2024-03-16T17:57:41.399406+0000","last_unstale":"2024-03-16T17:57:42.374104+0000","last_undegraded":"2024-03-16T17:57:42.374104+0000","last_fullsized":"2024-03-16T17:57:42.374104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376607+0000","last_change":"2024-03-16T17:57:41.416986+0000","last_active":"2024-03-16T17:57:42.376607+0000","last_peered":"2024-03-16T17:57:42.376607+0000","last_clean":"2024-03-16T17:57:42.376607+0000","last_became_active":"2024-03-16T17:57:41.416699+0000","last_became_peered":"2024-03-16T17:57:41.416699+0000","last_unstale":"2024-03-16T17:57:42.376607+0000","last_undegraded":"2024-03-16T17:57:42.376607+0000","last_fullsized":"2024-03-16T17:57:42.376607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:11:48.511692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733913+0000","last_change":"2024-03-16T17:57:41.394311+0000","last_active":"2024-03-16T17:57:42.733913+0000","last_peered":"2024-03-16T17:57:42.733913+0000","last_clean":"2024-03-16T17:57:42.733913+0000","last_became_active":"2024-03-16T17:57:41.393750+0000","last_became_peered":"2024-03-16T17:57:41.393750+0000","last_unstale":"2024-03-16T17:57:42.733913+0000","last_undegraded":"2024-03-16T17:57:42.733913+0000","last_fullsized":"2024-03-16T17:57:42.733913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:55:54.818895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736015+0000","last_change":"2024-03-16T17:57:41.408696+0000","last_active":"2024-03-16T17:57:42.736015+0000","last_peered":"2024-03-16T17:57:42.736015+0000","last_clean":"2024-03-16T17:57:42.736015+0000","last_became_active":"2024-03-16T17:57:41.407916+0000","last_became_peered":"2024-03-16T17:57:41.407916+0000","last_unstale":"2024-03-16T17:57:42.736015+0000","last_undegraded":"2024-03-16T17:57:42.736015+0000","last_fullsized":"2024-03-16T17:57:42.736015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:36:37.396828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733788+0000","last_change":"2024-03-16T17:57:41.396339+0000","last_active":"2024-03-16T17:57:42.733788+0000","last_peered":"2024-03-16T17:57:42.733788+0000","last_clean":"2024-03-16T17:57:42.733788+0000","last_became_active":"2024-03-16T17:57:41.396068+0000","last_became_peered":"2024-03-16T17:57:41.396068+0000","last_unstale":"2024-03-16T17:57:42.733788+0000","last_undegraded":"2024-03-16T17:57:42.733788+0000","last_fullsized":"2024-03-16T17:57:42.733788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732607+0000","last_change":"2024-03-16T17:57:41.390555+0000","last_active":"2024-03-16T17:57:42.732607+0000","last_peered":"2024-03-16T17:57:42.732607+0000","last_clean":"2024-03-16T17:57:42.732607+0000","last_became_active":"2024-03-16T17:57:41.389886+0000","last_became_peered":"2024-03-16T17:57:41.389886+0000","last_unstale":"2024-03-16T17:57:42.732607+0000","last_undegraded":"2024-03-16T17:57:42.732607+0000","last_fullsized":"2024-03-16T17:57:42.732607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:20.628509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375544+0000","last_change":"2024-03-16T17:57:41.411237+0000","last_active":"2024-03-16T17:57:42.375544+0000","last_peered":"2024-03-16T17:57:42.375544+0000","last_clean":"2024-03-16T17:57:42.375544+0000","last_became_active":"2024-03-16T17:57:41.410757+0000","last_became_peered":"2024-03-16T17:57:41.410757+0000","last_unstale":"2024-03-16T17:57:42.375544+0000","last_undegraded":"2024-03-16T17:57:42.375544+0000","last_fullsized":"2024-03-16T17:57:42.375544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:01:51.467418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733707+0000","last_change":"2024-03-16T17:57:41.391770+0000","last_active":"2024-03-16T17:57:42.733707+0000","last_peered":"2024-03-16T17:57:42.733707+0000","last_clean":"2024-03-16T17:57:42.733707+0000","last_became_active":"2024-03-16T17:57:41.391470+0000","last_became_peered":"2024-03-16T17:57:41.391470+0000","last_unstale":"2024-03-16T17:57:42.733707+0000","last_undegraded":"2024-03-16T17:57:42.733707+0000","last_fullsized":"2024-03-16T17:57:42.733707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374329+0000","last_change":"2024-03-16T17:57:41.400595+0000","last_active":"2024-03-16T17:57:42.374329+0000","last_peered":"2024-03-16T17:57:42.374329+0000","last_clean":"2024-03-16T17:57:42.374329+0000","last_became_active":"2024-03-16T17:57:41.400295+0000","last_became_peered":"2024-03-16T17:57:41.400295+0000","last_unstale":"2024-03-16T17:57:42.374329+0000","last_undegraded":"2024-03-16T17:57:42.374329+0000","last_fullsized":"2024-03-16T17:57:42.374329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:55:42.207131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734977+0000","last_change":"2024-03-16T17:57:41.391182+0000","last_active":"2024-03-16T17:57:42.734977+0000","last_peered":"2024-03-16T17:57:42.734977+0000","last_clean":"2024-03-16T17:57:42.734977+0000","last_became_active":"2024-03-16T17:57:41.390811+0000","last_became_peered":"2024-03-16T17:57:41.390811+0000","last_unstale":"2024-03-16T17:57:42.734977+0000","last_undegraded":"2024-03-16T17:57:42.734977+0000","last_fullsized":"2024-03-16T17:57:42.734977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:55:42.207131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734433+0000","last_change":"2024-03-16T17:57:41.407541+0000","last_active":"2024-03-16T17:57:42.734433+0000","last_peered":"2024-03-16T17:57:42.734433+0000","last_clean":"2024-03-16T17:57:42.734433+0000","last_became_active":"2024-03-16T17:57:41.405546+0000","last_became_peered":"2024-03-16T17:57:41.405546+0000","last_unstale":"2024-03-16T17:57:42.734433+0000","last_undegraded":"2024-03-16T17:57:42.734433+0000","last_fullsized":"2024-03-16T17:57:42.734433+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369831+0000","last_change":"2024-03-16T17:57:41.399134+0000","last_active":"2024-03-16T17:57:42.369831+0000","last_peered":"2024-03-16T17:57:42.369831+0000","last_clean":"2024-03-16T17:57:42.369831+0000","last_became_active":"2024-03-16T17:57:41.398841+0000","last_became_peered":"2024-03-16T17:57:41.398841+0000","last_unstale":"2024-03-16T17:57:42.369831+0000","last_undegraded":"2024-03-16T17:57:42.369831+0000","last_fullsized":"2024-03-16T17:57:42.369831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734641+0000","last_change":"2024-03-16T17:57:41.399114+0000","last_active":"2024-03-16T17:57:42.734641+0000","last_peered":"2024-03-16T17:57:42.734641+0000","last_clean":"2024-03-16T17:57:42.734641+0000","last_became_active":"2024-03-16T17:57:41.395668+0000","last_became_peered":"2024-03-16T17:57:41.395668+0000","last_unstale":"2024-03-16T17:57:42.734641+0000","last_undegraded":"2024-03-16T17:57:42.734641+0000","last_fullsized":"2024-03-16T17:57:42.734641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:41:26.597789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732560+0000","last_change":"2024-03-16T17:57:41.390678+0000","last_active":"2024-03-16T17:57:42.732560+0000","last_peered":"2024-03-16T17:57:42.732560+0000","last_clean":"2024-03-16T17:57:42.732560+0000","last_became_active":"2024-03-16T17:57:41.387968+0000","last_became_peered":"2024-03-16T17:57:41.387968+0000","last_unstale":"2024-03-16T17:57:42.732560+0000","last_undegraded":"2024-03-16T17:57:42.732560+0000","last_fullsized":"2024-03-16T17:57:42.732560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:59:12.277190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375711+0000","last_change":"2024-03-16T17:57:41.399083+0000","last_active":"2024-03-16T17:57:42.375711+0000","last_peered":"2024-03-16T17:57:42.375711+0000","last_clean":"2024-03-16T17:57:42.375711+0000","last_became_active":"2024-03-16T17:57:41.398899+0000","last_became_peered":"2024-03-16T17:57:41.398899+0000","last_unstale":"2024-03-16T17:57:42.375711+0000","last_undegraded":"2024-03-16T17:57:42.375711+0000","last_fullsized":"2024-03-16T17:57:42.375711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:10:39.722613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733367+0000","last_change":"2024-03-16T17:57:41.390700+0000","last_active":"2024-03-16T17:57:42.733367+0000","last_peered":"2024-03-16T17:57:42.733367+0000","last_clean":"2024-03-16T17:57:42.733367+0000","last_became_active":"2024-03-16T17:57:41.389732+0000","last_became_peered":"2024-03-16T17:57:41.389732+0000","last_unstale":"2024-03-16T17:57:42.733367+0000","last_undegraded":"2024-03-16T17:57:42.733367+0000","last_fullsized":"2024-03-16T17:57:42.733367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:32:52.837630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732990+0000","last_change":"2024-03-16T17:57:41.392698+0000","last_active":"2024-03-16T17:57:42.732990+0000","last_peered":"2024-03-16T17:57:42.732990+0000","last_clean":"2024-03-16T17:57:42.732990+0000","last_became_active":"2024-03-16T17:57:41.391950+0000","last_became_peered":"2024-03-16T17:57:41.391950+0000","last_unstale":"2024-03-16T17:57:42.732990+0000","last_undegraded":"2024-03-16T17:57:42.732990+0000","last_fullsized":"2024-03-16T17:57:42.732990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.372924+0000","last_change":"2024-03-16T17:57:41.383046+0000","last_active":"2024-03-16T17:57:42.372924+0000","last_peered":"2024-03-16T17:57:42.372924+0000","last_clean":"2024-03-16T17:57:42.372924+0000","last_became_active":"2024-03-16T17:57:41.382533+0000","last_became_peered":"2024-03-16T17:57:41.382533+0000","last_unstale":"2024-03-16T17:57:42.372924+0000","last_undegraded":"2024-03-16T17:57:42.372924+0000","last_fullsized":"2024-03-16T17:57:42.372924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:44:11.757790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734496+0000","last_change":"2024-03-16T17:57:40.368183+0000","last_active":"2024-03-16T17:57:42.734496+0000","last_peered":"2024-03-16T17:57:42.734496+0000","last_clean":"2024-03-16T17:57:42.734496+0000","last_became_active":"2024-03-16T17:57:40.367738+0000","last_became_peered":"2024-03-16T17:57:40.367738+0000","last_unstale":"2024-03-16T17:57:42.734496+0000","last_undegraded":"2024-03-16T17:57:42.734496+0000","last_fullsized":"2024-03-16T17:57:42.734496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732742+0000","last_change":"2024-03-16T17:57:41.378107+0000","last_active":"2024-03-16T17:57:42.732742+0000","last_peered":"2024-03-16T17:57:42.732742+0000","last_clean":"2024-03-16T17:57:42.732742+0000","last_became_active":"2024-03-16T17:57:41.375872+0000","last_became_peered":"2024-03-16T17:57:41.375872+0000","last_unstale":"2024-03-16T17:57:42.732742+0000","last_undegraded":"2024-03-16T17:57:42.732742+0000","last_fullsized":"2024-03-16T17:57:42.732742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733377+0000","last_change":"2024-03-16T17:57:40.375279+0000","last_active":"2024-03-16T17:57:42.733377+0000","last_peered":"2024-03-16T17:57:42.733377+0000","last_clean":"2024-03-16T17:57:42.733377+0000","last_became_active":"2024-03-16T17:57:40.374936+0000","last_became_peered":"2024-03-16T17:57:40.374936+0000","last_unstale":"2024-03-16T17:57:42.733377+0000","last_undegraded":"2024-03-16T17:57:42.733377+0000","last_fullsized":"2024-03-16T17:57:42.733377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:07.628213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733869+0000","last_change":"2024-03-16T17:57:40.372178+0000","last_active":"2024-03-16T17:57:42.733869+0000","last_peered":"2024-03-16T17:57:42.733869+0000","last_clean":"2024-03-16T17:57:42.733869+0000","last_became_active":"2024-03-16T17:57:40.370709+0000","last_became_peered":"2024-03-16T17:57:40.370709+0000","last_unstale":"2024-03-16T17:57:42.733869+0000","last_undegraded":"2024-03-16T17:57:42.733869+0000","last_fullsized":"2024-03-16T17:57:42.733869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:23:36.985797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732131+0000","last_change":"2024-03-16T17:57:41.397372+0000","last_active":"2024-03-16T17:57:42.732131+0000","last_peered":"2024-03-16T17:57:42.732131+0000","last_clean":"2024-03-16T17:57:42.732131+0000","last_became_active":"2024-03-16T17:57:41.397032+0000","last_became_peered":"2024-03-16T17:57:41.397032+0000","last_unstale":"2024-03-16T17:57:42.732131+0000","last_undegraded":"2024-03-16T17:57:42.732131+0000","last_fullsized":"2024-03-16T17:57:42.732131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374381+0000","last_change":"2024-03-16T17:57:40.362691+0000","last_active":"2024-03-16T17:57:42.374381+0000","last_peered":"2024-03-16T17:57:42.374381+0000","last_clean":"2024-03-16T17:57:42.374381+0000","last_became_active":"2024-03-16T17:57:40.361668+0000","last_became_peered":"2024-03-16T17:57:40.361668+0000","last_unstale":"2024-03-16T17:57:42.374381+0000","last_undegraded":"2024-03-16T17:57:42.374381+0000","last_fullsized":"2024-03-16T17:57:42.374381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369126+0000","last_change":"2024-03-16T17:57:41.393663+0000","last_active":"2024-03-16T17:57:42.369126+0000","last_peered":"2024-03-16T17:57:42.369126+0000","last_clean":"2024-03-16T17:57:42.369126+0000","last_became_active":"2024-03-16T17:57:41.393327+0000","last_became_peered":"2024-03-16T17:57:41.393327+0000","last_unstale":"2024-03-16T17:57:42.369126+0000","last_undegraded":"2024-03-16T17:57:42.369126+0000","last_fullsized":"2024-03-16T17:57:42.369126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733784+0000","last_change":"2024-03-16T17:57:40.377710+0000","last_active":"2024-03-16T17:57:42.733784+0000","last_peered":"2024-03-16T17:57:42.733784+0000","last_clean":"2024-03-16T17:57:42.733784+0000","last_became_active":"2024-03-16T17:57:40.377427+0000","last_became_peered":"2024-03-16T17:57:40.377427+0000","last_unstale":"2024-03-16T17:57:42.733784+0000","last_undegraded":"2024-03-16T17:57:42.733784+0000","last_fullsized":"2024-03-16T17:57:42.733784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376320+0000","last_change":"2024-03-16T17:57:41.413272+0000","last_active":"2024-03-16T17:57:42.376320+0000","last_peered":"2024-03-16T17:57:42.376320+0000","last_clean":"2024-03-16T17:57:42.376320+0000","last_became_active":"2024-03-16T17:57:41.412989+0000","last_became_peered":"2024-03-16T17:57:41.412989+0000","last_unstale":"2024-03-16T17:57:42.376320+0000","last_undegraded":"2024-03-16T17:57:42.376320+0000","last_fullsized":"2024-03-16T17:57:42.376320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:36:37.396828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376311+0000","last_change":"2024-03-16T17:57:40.380326+0000","last_active":"2024-03-16T17:57:42.376311+0000","last_peered":"2024-03-16T17:57:42.376311+0000","last_clean":"2024-03-16T17:57:42.376311+0000","last_became_active":"2024-03-16T17:57:40.380003+0000","last_became_peered":"2024-03-16T17:57:40.380003+0000","last_unstale":"2024-03-16T17:57:42.376311+0000","last_undegraded":"2024-03-16T17:57:42.376311+0000","last_fullsized":"2024-03-16T17:57:42.376311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:07.628213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733226+0000","last_change":"2024-03-16T17:57:41.407675+0000","last_active":"2024-03-16T17:57:42.733226+0000","last_peered":"2024-03-16T17:57:42.733226+0000","last_clean":"2024-03-16T17:57:42.733226+0000","last_became_active":"2024-03-16T17:57:41.407412+0000","last_became_peered":"2024-03-16T17:57:41.407412+0000","last_unstale":"2024-03-16T17:57:42.733226+0000","last_undegraded":"2024-03-16T17:57:42.733226+0000","last_fullsized":"2024-03-16T17:57:42.733226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370065+0000","last_change":"2024-03-16T17:57:40.375361+0000","last_active":"2024-03-16T17:57:42.370065+0000","last_peered":"2024-03-16T17:57:42.370065+0000","last_clean":"2024-03-16T17:57:42.370065+0000","last_became_active":"2024-03-16T17:57:40.375086+0000","last_became_peered":"2024-03-16T17:57:40.375086+0000","last_unstale":"2024-03-16T17:57:42.370065+0000","last_undegraded":"2024-03-16T17:57:42.370065+0000","last_fullsized":"2024-03-16T17:57:42.370065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373846+0000","last_change":"2024-03-16T17:57:41.404551+0000","last_active":"2024-03-16T17:57:42.373846+0000","last_peered":"2024-03-16T17:57:42.373846+0000","last_clean":"2024-03-16T17:57:42.373846+0000","last_became_active":"2024-03-16T17:57:41.404266+0000","last_became_peered":"2024-03-16T17:57:41.404266+0000","last_unstale":"2024-03-16T17:57:42.373846+0000","last_undegraded":"2024-03-16T17:57:42.373846+0000","last_fullsized":"2024-03-16T17:57:42.373846+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:55:54.818895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733842+0000","last_change":"2024-03-16T17:57:40.376223+0000","last_active":"2024-03-16T17:57:42.733842+0000","last_peered":"2024-03-16T17:57:42.733842+0000","last_clean":"2024-03-16T17:57:42.733842+0000","last_became_active":"2024-03-16T17:57:40.375873+0000","last_became_peered":"2024-03-16T17:57:40.375873+0000","last_unstale":"2024-03-16T17:57:42.733842+0000","last_undegraded":"2024-03-16T17:57:42.733842+0000","last_fullsized":"2024-03-16T17:57:42.733842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735456+0000","last_change":"2024-03-16T17:57:41.403063+0000","last_active":"2024-03-16T17:57:42.735456+0000","last_peered":"2024-03-16T17:57:42.735456+0000","last_clean":"2024-03-16T17:57:42.735456+0000","last_became_active":"2024-03-16T17:57:41.402736+0000","last_became_peered":"2024-03-16T17:57:41.402736+0000","last_unstale":"2024-03-16T17:57:42.735456+0000","last_undegraded":"2024-03-16T17:57:42.735456+0000","last_fullsized":"2024-03-16T17:57:42.735456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:10:39.722613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732825+0000","last_change":"2024-03-16T17:57:40.363660+0000","last_active":"2024-03-16T17:57:42.732825+0000","last_peered":"2024-03-16T17:57:42.732825+0000","last_clean":"2024-03-16T17:57:42.732825+0000","last_became_active":"2024-03-16T17:57:40.363245+0000","last_became_peered":"2024-03-16T17:57:40.363245+0000","last_unstale":"2024-03-16T17:57:42.732825+0000","last_undegraded":"2024-03-16T17:57:42.732825+0000","last_fullsized":"2024-03-16T17:57:42.732825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:45:57.586553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732613+0000","last_change":"2024-03-16T17:57:41.398106+0000","last_active":"2024-03-16T17:57:42.732613+0000","last_peered":"2024-03-16T17:57:42.732613+0000","last_clean":"2024-03-16T17:57:42.732613+0000","last_became_active":"2024-03-16T17:57:41.396446+0000","last_became_peered":"2024-03-16T17:57:41.396446+0000","last_unstale":"2024-03-16T17:57:42.732613+0000","last_undegraded":"2024-03-16T17:57:42.732613+0000","last_fullsized":"2024-03-16T17:57:42.732613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:42:47.316038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375786+0000","last_change":"2024-03-16T17:57:40.377535+0000","last_active":"2024-03-16T17:57:42.375786+0000","last_peered":"2024-03-16T17:57:42.375786+0000","last_clean":"2024-03-16T17:57:42.375786+0000","last_became_active":"2024-03-16T17:57:40.377105+0000","last_became_peered":"2024-03-16T17:57:40.377105+0000","last_unstale":"2024-03-16T17:57:42.375786+0000","last_undegraded":"2024-03-16T17:57:42.375786+0000","last_fullsized":"2024-03-16T17:57:42.375786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735111+0000","last_change":"2024-03-16T17:57:41.407322+0000","last_active":"2024-03-16T17:57:42.735111+0000","last_peered":"2024-03-16T17:57:42.735111+0000","last_clean":"2024-03-16T17:57:42.735111+0000","last_became_active":"2024-03-16T17:57:41.403879+0000","last_became_peered":"2024-03-16T17:57:41.403879+0000","last_unstale":"2024-03-16T17:57:42.735111+0000","last_undegraded":"2024-03-16T17:57:42.735111+0000","last_fullsized":"2024-03-16T17:57:42.735111+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:16:17.454713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732837+0000","last_change":"2024-03-16T17:57:40.368238+0000","last_active":"2024-03-16T17:57:42.732837+0000","last_peered":"2024-03-16T17:57:42.732837+0000","last_clean":"2024-03-16T17:57:42.732837+0000","last_became_active":"2024-03-16T17:57:40.367367+0000","last_became_peered":"2024-03-16T17:57:40.367367+0000","last_unstale":"2024-03-16T17:57:42.732837+0000","last_undegraded":"2024-03-16T17:57:42.732837+0000","last_fullsized":"2024-03-16T17:57:42.732837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:26.104046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732994+0000","last_change":"2024-03-16T17:57:41.389244+0000","last_active":"2024-03-16T17:57:42.732994+0000","last_peered":"2024-03-16T17:57:42.732994+0000","last_clean":"2024-03-16T17:57:42.732994+0000","last_became_active":"2024-03-16T17:57:41.388858+0000","last_became_peered":"2024-03-16T17:57:41.388858+0000","last_unstale":"2024-03-16T17:57:42.732994+0000","last_undegraded":"2024-03-16T17:57:42.732994+0000","last_fullsized":"2024-03-16T17:57:42.732994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:21:18.681255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733311+0000","last_change":"2024-03-16T17:57:40.365096+0000","last_active":"2024-03-16T17:57:42.733311+0000","last_peered":"2024-03-16T17:57:42.733311+0000","last_clean":"2024-03-16T17:57:42.733311+0000","last_became_active":"2024-03-16T17:57:40.364700+0000","last_became_peered":"2024-03-16T17:57:40.364700+0000","last_unstale":"2024-03-16T17:57:42.733311+0000","last_undegraded":"2024-03-16T17:57:42.733311+0000","last_fullsized":"2024-03-16T17:57:42.733311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:53:51.562074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734633+0000","last_change":"2024-03-16T17:57:41.405056+0000","last_active":"2024-03-16T17:57:42.734633+0000","last_peered":"2024-03-16T17:57:42.734633+0000","last_clean":"2024-03-16T17:57:42.734633+0000","last_became_active":"2024-03-16T17:57:41.403286+0000","last_became_peered":"2024-03-16T17:57:41.403286+0000","last_unstale":"2024-03-16T17:57:42.734633+0000","last_undegraded":"2024-03-16T17:57:42.734633+0000","last_fullsized":"2024-03-16T17:57:42.734633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374263+0000","last_change":"2024-03-16T17:57:40.365256+0000","last_active":"2024-03-16T17:57:42.374263+0000","last_peered":"2024-03-16T17:57:42.374263+0000","last_clean":"2024-03-16T17:57:42.374263+0000","last_became_active":"2024-03-16T17:57:40.362214+0000","last_became_peered":"2024-03-16T17:57:40.362214+0000","last_unstale":"2024-03-16T17:57:42.374263+0000","last_undegraded":"2024-03-16T17:57:42.374263+0000","last_fullsized":"2024-03-16T17:57:42.374263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:22.039976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735167+0000","last_change":"2024-03-16T17:57:41.413401+0000","last_active":"2024-03-16T17:57:42.735167+0000","last_peered":"2024-03-16T17:57:42.735167+0000","last_clean":"2024-03-16T17:57:42.735167+0000","last_became_active":"2024-03-16T17:57:41.413163+0000","last_became_peered":"2024-03-16T17:57:41.413163+0000","last_unstale":"2024-03-16T17:57:42.735167+0000","last_undegraded":"2024-03-16T17:57:42.735167+0000","last_fullsized":"2024-03-16T17:57:42.735167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:01:51.467418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734097+0000","last_change":"2024-03-16T17:57:40.370143+0000","last_active":"2024-03-16T17:57:42.734097+0000","last_peered":"2024-03-16T17:57:42.734097+0000","last_clean":"2024-03-16T17:57:42.734097+0000","last_became_active":"2024-03-16T17:57:40.369743+0000","last_became_peered":"2024-03-16T17:57:40.369743+0000","last_unstale":"2024-03-16T17:57:42.734097+0000","last_undegraded":"2024-03-16T17:57:42.734097+0000","last_fullsized":"2024-03-16T17:57:42.734097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:32:24.057779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736551+0000","last_change":"2024-03-16T17:57:41.413116+0000","last_active":"2024-03-16T17:57:42.736551+0000","last_peered":"2024-03-16T17:57:42.736551+0000","last_clean":"2024-03-16T17:57:42.736551+0000","last_became_active":"2024-03-16T17:57:41.412868+0000","last_became_peered":"2024-03-16T17:57:41.412868+0000","last_unstale":"2024-03-16T17:57:42.736551+0000","last_undegraded":"2024-03-16T17:57:42.736551+0000","last_fullsized":"2024-03-16T17:57:42.736551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:20:56.534791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.370370+0000","last_change":"2024-03-16T17:57:40.364807+0000","last_active":"2024-03-16T17:57:42.370370+0000","last_peered":"2024-03-16T17:57:42.370370+0000","last_clean":"2024-03-16T17:57:42.370370+0000","last_became_active":"2024-03-16T17:57:40.363383+0000","last_became_peered":"2024-03-16T17:57:40.363383+0000","last_unstale":"2024-03-16T17:57:42.370370+0000","last_undegraded":"2024-03-16T17:57:42.370370+0000","last_fullsized":"2024-03-16T17:57:42.370370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:16.023603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375496+0000","last_change":"2024-03-16T17:57:41.402990+0000","last_active":"2024-03-16T17:57:42.375496+0000","last_peered":"2024-03-16T17:57:42.375496+0000","last_clean":"2024-03-16T17:57:42.375496+0000","last_became_active":"2024-03-16T17:57:41.402696+0000","last_became_peered":"2024-03-16T17:57:41.402696+0000","last_unstale":"2024-03-16T17:57:42.375496+0000","last_undegraded":"2024-03-16T17:57:42.375496+0000","last_fullsized":"2024-03-16T17:57:42.375496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:16:17.454713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733543+0000","last_change":"2024-03-16T17:57:40.378607+0000","last_active":"2024-03-16T17:57:42.733543+0000","last_peered":"2024-03-16T17:57:42.733543+0000","last_clean":"2024-03-16T17:57:42.733543+0000","last_became_active":"2024-03-16T17:57:40.378291+0000","last_became_peered":"2024-03-16T17:57:40.378291+0000","last_unstale":"2024-03-16T17:57:42.733543+0000","last_undegraded":"2024-03-16T17:57:42.733543+0000","last_fullsized":"2024-03-16T17:57:42.733543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:16.771276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735929+0000","last_change":"2024-03-16T17:57:41.409430+0000","last_active":"2024-03-16T17:57:42.735929+0000","last_peered":"2024-03-16T17:57:42.735929+0000","last_clean":"2024-03-16T17:57:42.735929+0000","last_became_active":"2024-03-16T17:57:41.409209+0000","last_became_peered":"2024-03-16T17:57:41.409209+0000","last_unstale":"2024-03-16T17:57:42.735929+0000","last_undegraded":"2024-03-16T17:57:42.735929+0000","last_fullsized":"2024-03-16T17:57:42.735929+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:58.061341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733228+0000","last_change":"2024-03-16T17:57:35.560172+0000","last_active":"2024-03-16T17:57:42.733228+0000","last_peered":"2024-03-16T17:57:42.733228+0000","last_clean":"2024-03-16T17:57:42.733228+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T17:57:42.733228+0000","last_undegraded":"2024-03-16T17:57:42.733228+0000","last_fullsized":"2024-03-16T17:57:42.733228+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_clean_scrub_stamp":"2024-03-16T17:57:34.174437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:20:03.039404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.374626+0000","last_change":"2024-03-16T17:57:40.372426+0000","last_active":"2024-03-16T17:57:42.374626+0000","last_peered":"2024-03-16T17:57:42.374626+0000","last_clean":"2024-03-16T17:57:42.374626+0000","last_became_active":"2024-03-16T17:57:40.372097+0000","last_became_peered":"2024-03-16T17:57:40.372097+0000","last_unstale":"2024-03-16T17:57:42.374626+0000","last_undegraded":"2024-03-16T17:57:42.374626+0000","last_fullsized":"2024-03-16T17:57:42.374626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734031+0000","last_change":"2024-03-16T17:57:41.401095+0000","last_active":"2024-03-16T17:57:42.734031+0000","last_peered":"2024-03-16T17:57:42.734031+0000","last_clean":"2024-03-16T17:57:42.734031+0000","last_became_active":"2024-03-16T17:57:41.400788+0000","last_became_peered":"2024-03-16T17:57:41.400788+0000","last_unstale":"2024-03-16T17:57:42.734031+0000","last_undegraded":"2024-03-16T17:57:42.734031+0000","last_fullsized":"2024-03-16T17:57:42.734031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733259+0000","last_change":"2024-03-16T17:57:40.369748+0000","last_active":"2024-03-16T17:57:42.733259+0000","last_peered":"2024-03-16T17:57:42.733259+0000","last_clean":"2024-03-16T17:57:42.733259+0000","last_became_active":"2024-03-16T17:57:40.368850+0000","last_became_peered":"2024-03-16T17:57:40.368850+0000","last_unstale":"2024-03-16T17:57:42.733259+0000","last_undegraded":"2024-03-16T17:57:42.733259+0000","last_fullsized":"2024-03-16T17:57:42.733259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:31:16.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732283+0000","last_change":"2024-03-16T17:57:41.373872+0000","last_active":"2024-03-16T17:57:42.732283+0000","last_peered":"2024-03-16T17:57:42.732283+0000","last_clean":"2024-03-16T17:57:42.732283+0000","last_became_active":"2024-03-16T17:57:41.373478+0000","last_became_peered":"2024-03-16T17:57:41.373478+0000","last_unstale":"2024-03-16T17:57:42.732283+0000","last_undegraded":"2024-03-16T17:57:42.732283+0000","last_fullsized":"2024-03-16T17:57:42.732283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:04:23.189434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733503+0000","last_change":"2024-03-16T17:57:40.373316+0000","last_active":"2024-03-16T17:57:42.733503+0000","last_peered":"2024-03-16T17:57:42.733503+0000","last_clean":"2024-03-16T17:57:42.733503+0000","last_became_active":"2024-03-16T17:57:40.373015+0000","last_became_peered":"2024-03-16T17:57:40.373015+0000","last_unstale":"2024-03-16T17:57:42.733503+0000","last_undegraded":"2024-03-16T17:57:42.733503+0000","last_fullsized":"2024-03-16T17:57:42.733503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:42:38.164765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369425+0000","last_change":"2024-03-16T17:57:41.383198+0000","last_active":"2024-03-16T17:57:42.369425+0000","last_peered":"2024-03-16T17:57:42.369425+0000","last_clean":"2024-03-16T17:57:42.369425+0000","last_became_active":"2024-03-16T17:57:41.382886+0000","last_became_peered":"2024-03-16T17:57:41.382886+0000","last_unstale":"2024-03-16T17:57:42.369425+0000","last_undegraded":"2024-03-16T17:57:42.369425+0000","last_fullsized":"2024-03-16T17:57:42.369425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375951+0000","last_change":"2024-03-16T17:57:40.368934+0000","last_active":"2024-03-16T17:57:42.375951+0000","last_peered":"2024-03-16T17:57:42.375951+0000","last_clean":"2024-03-16T17:57:42.375951+0000","last_became_active":"2024-03-16T17:57:40.368496+0000","last_became_peered":"2024-03-16T17:57:40.368496+0000","last_unstale":"2024-03-16T17:57:42.375951+0000","last_undegraded":"2024-03-16T17:57:42.375951+0000","last_fullsized":"2024-03-16T17:57:42.375951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:40:06.480561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.373158+0000","last_change":"2024-03-16T17:57:41.377235+0000","last_active":"2024-03-16T17:57:42.373158+0000","last_peered":"2024-03-16T17:57:42.373158+0000","last_clean":"2024-03-16T17:57:42.373158+0000","last_became_active":"2024-03-16T17:57:41.376796+0000","last_became_peered":"2024-03-16T17:57:41.376796+0000","last_unstale":"2024-03-16T17:57:42.373158+0000","last_undegraded":"2024-03-16T17:57:42.373158+0000","last_fullsized":"2024-03-16T17:57:42.373158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:24:23.606350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369894+0000","last_change":"2024-03-16T17:57:40.365747+0000","last_active":"2024-03-16T17:57:42.369894+0000","last_peered":"2024-03-16T17:57:42.369894+0000","last_clean":"2024-03-16T17:57:42.369894+0000","last_became_active":"2024-03-16T17:57:40.364579+0000","last_became_peered":"2024-03-16T17:57:40.364579+0000","last_unstale":"2024-03-16T17:57:42.369894+0000","last_undegraded":"2024-03-16T17:57:42.369894+0000","last_fullsized":"2024-03-16T17:57:42.369894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376270+0000","last_change":"2024-03-16T17:57:41.404470+0000","last_active":"2024-03-16T17:57:42.376270+0000","last_peered":"2024-03-16T17:57:42.376270+0000","last_clean":"2024-03-16T17:57:42.376270+0000","last_became_active":"2024-03-16T17:57:41.404100+0000","last_became_peered":"2024-03-16T17:57:41.404100+0000","last_unstale":"2024-03-16T17:57:42.376270+0000","last_undegraded":"2024-03-16T17:57:42.376270+0000","last_fullsized":"2024-03-16T17:57:42.376270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:50:09.742348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.736227+0000","last_change":"2024-03-16T17:57:40.370118+0000","last_active":"2024-03-16T17:57:42.736227+0000","last_peered":"2024-03-16T17:57:42.736227+0000","last_clean":"2024-03-16T17:57:42.736227+0000","last_became_active":"2024-03-16T17:57:40.369751+0000","last_became_peered":"2024-03-16T17:57:40.369751+0000","last_unstale":"2024-03-16T17:57:42.736227+0000","last_undegraded":"2024-03-16T17:57:42.736227+0000","last_fullsized":"2024-03-16T17:57:42.736227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:04:22.171852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369793+0000","last_change":"2024-03-16T17:57:41.384659+0000","last_active":"2024-03-16T17:57:42.369793+0000","last_peered":"2024-03-16T17:57:42.369793+0000","last_clean":"2024-03-16T17:57:42.369793+0000","last_became_active":"2024-03-16T17:57:41.384374+0000","last_became_peered":"2024-03-16T17:57:41.384374+0000","last_unstale":"2024-03-16T17:57:42.369793+0000","last_undegraded":"2024-03-16T17:57:42.369793+0000","last_fullsized":"2024-03-16T17:57:42.369793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:39:50.214879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:43.209075+0000","last_change":"2024-03-16T17:57:40.377982+0000","last_active":"2024-03-16T17:57:43.209075+0000","last_peered":"2024-03-16T17:57:43.209075+0000","last_clean":"2024-03-16T17:57:43.209075+0000","last_became_active":"2024-03-16T17:57:40.377669+0000","last_became_peered":"2024-03-16T17:57:40.377669+0000","last_unstale":"2024-03-16T17:57:43.209075+0000","last_undegraded":"2024-03-16T17:57:43.209075+0000","last_fullsized":"2024-03-16T17:57:43.209075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:31:16.409174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735021+0000","last_change":"2024-03-16T17:57:41.400625+0000","last_active":"2024-03-16T17:57:42.735021+0000","last_peered":"2024-03-16T17:57:42.735021+0000","last_clean":"2024-03-16T17:57:42.735021+0000","last_became_active":"2024-03-16T17:57:41.398361+0000","last_became_peered":"2024-03-16T17:57:41.398361+0000","last_unstale":"2024-03-16T17:57:42.735021+0000","last_undegraded":"2024-03-16T17:57:42.735021+0000","last_fullsized":"2024-03-16T17:57:42.735021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733837+0000","last_change":"2024-03-16T17:57:40.379426+0000","last_active":"2024-03-16T17:57:42.733837+0000","last_peered":"2024-03-16T17:57:42.733837+0000","last_clean":"2024-03-16T17:57:42.733837+0000","last_became_active":"2024-03-16T17:57:40.379136+0000","last_became_peered":"2024-03-16T17:57:40.379136+0000","last_unstale":"2024-03-16T17:57:42.733837+0000","last_undegraded":"2024-03-16T17:57:42.733837+0000","last_fullsized":"2024-03-16T17:57:42.733837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:47:08.915987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732957+0000","last_change":"2024-03-16T17:57:41.379286+0000","last_active":"2024-03-16T17:57:42.732957+0000","last_peered":"2024-03-16T17:57:42.732957+0000","last_clean":"2024-03-16T17:57:42.732957+0000","last_became_active":"2024-03-16T17:57:41.378998+0000","last_became_peered":"2024-03-16T17:57:41.378998+0000","last_unstale":"2024-03-16T17:57:42.732957+0000","last_undegraded":"2024-03-16T17:57:42.732957+0000","last_fullsized":"2024-03-16T17:57:42.732957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732976+0000","last_change":"2024-03-16T17:57:40.369141+0000","last_active":"2024-03-16T17:57:42.732976+0000","last_peered":"2024-03-16T17:57:42.732976+0000","last_clean":"2024-03-16T17:57:42.732976+0000","last_became_active":"2024-03-16T17:57:40.368542+0000","last_became_peered":"2024-03-16T17:57:40.368542+0000","last_unstale":"2024-03-16T17:57:42.732976+0000","last_undegraded":"2024-03-16T17:57:42.732976+0000","last_fullsized":"2024-03-16T17:57:42.732976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.372617+0000","last_change":"2024-03-16T17:57:41.375830+0000","last_active":"2024-03-16T17:57:42.372617+0000","last_peered":"2024-03-16T17:57:42.372617+0000","last_clean":"2024-03-16T17:57:42.372617+0000","last_became_active":"2024-03-16T17:57:41.375515+0000","last_became_peered":"2024-03-16T17:57:41.375515+0000","last_unstale":"2024-03-16T17:57:42.372617+0000","last_undegraded":"2024-03-16T17:57:42.372617+0000","last_fullsized":"2024-03-16T17:57:42.372617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:32:25.075362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369244+0000","last_change":"2024-03-16T17:57:40.367142+0000","last_active":"2024-03-16T17:57:42.369244+0000","last_peered":"2024-03-16T17:57:42.369244+0000","last_clean":"2024-03-16T17:57:42.369244+0000","last_became_active":"2024-03-16T17:57:40.366778+0000","last_became_peered":"2024-03-16T17:57:40.366778+0000","last_unstale":"2024-03-16T17:57:42.369244+0000","last_undegraded":"2024-03-16T17:57:42.369244+0000","last_fullsized":"2024-03-16T17:57:42.369244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732297+0000","last_change":"2024-03-16T17:57:41.388886+0000","last_active":"2024-03-16T17:57:42.732297+0000","last_peered":"2024-03-16T17:57:42.732297+0000","last_clean":"2024-03-16T17:57:42.732297+0000","last_became_active":"2024-03-16T17:57:41.388588+0000","last_became_peered":"2024-03-16T17:57:41.388588+0000","last_unstale":"2024-03-16T17:57:42.732297+0000","last_undegraded":"2024-03-16T17:57:42.732297+0000","last_fullsized":"2024-03-16T17:57:42.732297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:24:00.217043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.369599+0000","last_change":"2024-03-16T17:57:40.371549+0000","last_active":"2024-03-16T17:57:42.369599+0000","last_peered":"2024-03-16T17:57:42.369599+0000","last_clean":"2024-03-16T17:57:42.369599+0000","last_became_active":"2024-03-16T17:57:40.371257+0000","last_became_peered":"2024-03-16T17:57:40.371257+0000","last_unstale":"2024-03-16T17:57:42.369599+0000","last_undegraded":"2024-03-16T17:57:42.369599+0000","last_fullsized":"2024-03-16T17:57:42.369599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734385+0000","last_change":"2024-03-16T17:57:41.392961+0000","last_active":"2024-03-16T17:57:42.734385+0000","last_peered":"2024-03-16T17:57:42.734385+0000","last_clean":"2024-03-16T17:57:42.734385+0000","last_became_active":"2024-03-16T17:57:41.392665+0000","last_became_peered":"2024-03-16T17:57:41.392665+0000","last_unstale":"2024-03-16T17:57:42.734385+0000","last_undegraded":"2024-03-16T17:57:42.734385+0000","last_fullsized":"2024-03-16T17:57:42.734385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:56:29.339126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735443+0000","last_change":"2024-03-16T17:57:40.376135+0000","last_active":"2024-03-16T17:57:42.735443+0000","last_peered":"2024-03-16T17:57:42.735443+0000","last_clean":"2024-03-16T17:57:42.735443+0000","last_became_active":"2024-03-16T17:57:40.375292+0000","last_became_peered":"2024-03-16T17:57:40.375292+0000","last_unstale":"2024-03-16T17:57:42.735443+0000","last_undegraded":"2024-03-16T17:57:42.735443+0000","last_fullsized":"2024-03-16T17:57:42.735443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:16.023603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733288+0000","last_change":"2024-03-16T17:57:41.394268+0000","last_active":"2024-03-16T17:57:42.733288+0000","last_peered":"2024-03-16T17:57:42.733288+0000","last_clean":"2024-03-16T17:57:42.733288+0000","last_became_active":"2024-03-16T17:57:41.393941+0000","last_became_peered":"2024-03-16T17:57:41.393941+0000","last_unstale":"2024-03-16T17:57:42.733288+0000","last_undegraded":"2024-03-16T17:57:42.733288+0000","last_fullsized":"2024-03-16T17:57:42.733288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:34:55.768329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.376119+0000","last_change":"2024-03-16T17:57:41.402175+0000","last_active":"2024-03-16T17:57:42.376119+0000","last_peered":"2024-03-16T17:57:42.376119+0000","last_clean":"2024-03-16T17:57:42.376119+0000","last_became_active":"2024-03-16T17:57:41.401851+0000","last_became_peered":"2024-03-16T17:57:41.401851+0000","last_unstale":"2024-03-16T17:57:42.376119+0000","last_undegraded":"2024-03-16T17:57:42.376119+0000","last_fullsized":"2024-03-16T17:57:42.376119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:49:58.061341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734977+0000","last_change":"2024-03-16T17:57:40.377332+0000","last_active":"2024-03-16T17:57:42.734977+0000","last_peered":"2024-03-16T17:57:42.734977+0000","last_clean":"2024-03-16T17:57:42.734977+0000","last_became_active":"2024-03-16T17:57:40.377028+0000","last_became_peered":"2024-03-16T17:57:40.377028+0000","last_unstale":"2024-03-16T17:57:42.734977+0000","last_undegraded":"2024-03-16T17:57:42.734977+0000","last_fullsized":"2024-03-16T17:57:42.734977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:04.556728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.375244+0000","last_change":"2024-03-16T17:57:41.402689+0000","last_active":"2024-03-16T17:57:42.375244+0000","last_peered":"2024-03-16T17:57:42.375244+0000","last_clean":"2024-03-16T17:57:42.375244+0000","last_became_active":"2024-03-16T17:57:41.401253+0000","last_became_peered":"2024-03-16T17:57:41.401253+0000","last_unstale":"2024-03-16T17:57:42.375244+0000","last_undegraded":"2024-03-16T17:57:42.375244+0000","last_fullsized":"2024-03-16T17:57:42.375244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:02:08.420204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735773+0000","last_change":"2024-03-16T17:57:40.379396+0000","last_active":"2024-03-16T17:57:42.735773+0000","last_peered":"2024-03-16T17:57:42.735773+0000","last_clean":"2024-03-16T17:57:42.735773+0000","last_became_active":"2024-03-16T17:57:40.379058+0000","last_became_peered":"2024-03-16T17:57:40.379058+0000","last_unstale":"2024-03-16T17:57:42.735773+0000","last_undegraded":"2024-03-16T17:57:42.735773+0000","last_fullsized":"2024-03-16T17:57:42.735773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:53:59.451003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.732999+0000","last_change":"2024-03-16T17:57:41.407792+0000","last_active":"2024-03-16T17:57:42.732999+0000","last_peered":"2024-03-16T17:57:42.732999+0000","last_clean":"2024-03-16T17:57:42.732999+0000","last_became_active":"2024-03-16T17:57:41.407412+0000","last_became_peered":"2024-03-16T17:57:41.407412+0000","last_unstale":"2024-03-16T17:57:42.732999+0000","last_undegraded":"2024-03-16T17:57:42.732999+0000","last_fullsized":"2024-03-16T17:57:42.732999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:30:06.813604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.735117+0000","last_change":"2024-03-16T17:57:40.366569+0000","last_active":"2024-03-16T17:57:42.735117+0000","last_peered":"2024-03-16T17:57:42.735117+0000","last_clean":"2024-03-16T17:57:42.735117+0000","last_became_active":"2024-03-16T17:57:40.362328+0000","last_became_peered":"2024-03-16T17:57:40.362328+0000","last_unstale":"2024-03-16T17:57:42.735117+0000","last_undegraded":"2024-03-16T17:57:42.735117+0000","last_fullsized":"2024-03-16T17:57:42.735117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:19:12.115370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.734985+0000","last_change":"2024-03-16T17:57:41.400212+0000","last_active":"2024-03-16T17:57:42.734985+0000","last_peered":"2024-03-16T17:57:42.734985+0000","last_clean":"2024-03-16T17:57:42.734985+0000","last_became_active":"2024-03-16T17:57:41.399937+0000","last_became_peered":"2024-03-16T17:57:41.399937+0000","last_unstale":"2024-03-16T17:57:42.734985+0000","last_undegraded":"2024-03-16T17:57:42.734985+0000","last_fullsized":"2024-03-16T17:57:42.734985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:40.329185+0000","last_clean_scrub_stamp":"2024-03-16T17:57:40.329185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:11:53.534693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T17:57:42.733309+0000","last_change":"2024-03-16T17:57:40.368635+0000","last_active":"2024-03-16T17:57:42.733309+0000","last_peered":"2024-03-16T17:57:42.733309+0000","last_clean":"2024-03-16T17:57:42.733309+0000","last_became_active":"2024-03-16T17:57:40.368370+0000","last_became_peered":"2024-03-16T17:57:40.368370+0000","last_unstale":"2024-03-16T17:57:42.733309+0000","last_undegraded":"2024-03-16T17:57:42.733309+0000","last_fullsized":"2024-03-16T17:57:42.733309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:39.311603+0000","last_clean_scrub_stamp":"2024-03-16T17:57:39.311603+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:31:19.918231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":665113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":74253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":75775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":74163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":74323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":665169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T17:57:51.758 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-16T17:57:51.758 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-16T17:57:51.758 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-16T17:57:51.759 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-16T17:57:52.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.997+0000 7fe1bd000640 1 Processor -- start 2024-03-16T17:57:52.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.998+0000 7fe1bd000640 1 -- start start 2024-03-16T17:57:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.999+0000 7fe1bd000640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe1b8143e00 0x7fe1b8070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.999+0000 7fe1bd000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1b80cd660 0x7fe1b80cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.999+0000 7fe1bd000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80cf3a0 con 0x7fe1b8070820 2024-03-16T17:57:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.999+0000 7fe1bd000640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b8071140 con 0x7fe1b8143e00 2024-03-16T17:57:52.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:51.999+0000 7fe1bd000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80cf610 con 0x7fe1b80cd660 2024-03-16T17:57:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.000+0000 7fe1afe00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe1b8143e00 0x7fe1b8070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.000+0000 7fe1afe00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe1b8143e00 0x7fe1b8070c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:60660/0 (socket says 172.21.15.82:60660) 2024-03-16T17:57:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.000+0000 7fe1afe00640 1 -- 172.21.15.82:0/692479851 learned_addr learned my addr 172.21.15.82:0/692479851 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.000+0000 7fe1b7e00640 1 --2- 172.21.15.82:0/692479851 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1b80cd660 0x7fe1b80cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.001+0000 7fe1afe00640 1 -- 172.21.15.82:0/692479851 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1b80cd660 msgr2=0x7fe1b80cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.001+0000 7fe1afe00640 1 --2- 172.21.15.82:0/692479851 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1b80cd660 0x7fe1b80cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.001+0000 7fe1afe00640 1 -- 172.21.15.82:0/692479851 >> v1:172.21.15.90:6789/0 conn(0x7fe1b8070820 legacy=0x7fe1b815f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.001+0000 7fe1afe00640 1 -- 172.21.15.82:0/692479851 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b80cf8f0 con 0x7fe1b8143e00 2024-03-16T17:57:52.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.002+0000 7fe1afe00640 1 --2- 172.21.15.82:0/692479851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1b8143e00 0x7fe1b8070c00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009870 tx=0x7fe1a002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24325efbbd2acfa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:52.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.002+0000 7fe1b6a00640 1 -- 172.21.15.82:0/692479851 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0031640 con 0x7fe1b8143e00 2024-03-16T17:57:52.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.002+0000 7fe1b6a00640 1 -- 172.21.15.82:0/692479851 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a00317a0 con 0x7fe1b8143e00 2024-03-16T17:57:52.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.002+0000 7fe1b6a00640 1 -- 172.21.15.82:0/692479851 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0031b30 con 0x7fe1b8143e00 2024-03-16T17:57:52.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.003+0000 7fe1bd000640 1 -- 172.21.15.82:0/692479851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1b8143e00 msgr2=0x7fe1b8070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.003+0000 7fe1bd000640 1 --2- 172.21.15.82:0/692479851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1b8143e00 0x7fe1b8070c00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009870 tx=0x7fe1a002b8f0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.003+0000 7fe1bd000640 1 -- 172.21.15.82:0/692479851 shutdown_connections 2024-03-16T17:57:52.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.003+0000 7fe1bd000640 1 --2- 172.21.15.82:0/692479851 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1b80cd660 0x7fe1b80cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.003+0000 7fe1bd000640 1 --2- 172.21.15.82:0/692479851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1b8143e00 0x7fe1b8070c00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.003+0000 7fe1bd000640 1 -- 172.21.15.82:0/692479851 >> 172.21.15.82:0/692479851 conn(0x7fe1b806ae70 msgr2=0x7fe1b806b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:52.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.003+0000 7fe1bd000640 1 -- 172.21.15.82:0/692479851 shutdown_connections 2024-03-16T17:57:52.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.003+0000 7fe1bd000640 1 -- 172.21.15.82:0/692479851 wait complete. 2024-03-16T17:57:52.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.005+0000 7fe1bd000640 1 Processor -- start 2024-03-16T17:57:52.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.005+0000 7fe1bd000640 1 -- start start 2024-03-16T17:57:52.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1bd000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1b8070820 0x7fe1b81f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1bd000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b80cd660 0x7fe1b81f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1bd000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1b8143e00 0x7fe1b813e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1bd000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b813bf50 con 0x7fe1b80cd660 2024-03-16T17:57:52.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1bd000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b813c160 con 0x7fe1b8070820 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1bd000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b813c440 con 0x7fe1b8143e00 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1afe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b80cd660 0x7fe1b81f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1afe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b80cd660 0x7fe1b81f42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35520/0 (socket says 172.21.15.82:35520) 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1afe00640 1 -- 172.21.15.82:0/1558723465 learned_addr learned my addr 172.21.15.82:0/1558723465 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1b7400640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1b8070820 0x7fe1b81f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.006+0000 7fe1b7e00640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1b8143e00 0x7fe1b813e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1afe00640 1 -- 172.21.15.82:0/1558723465 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1b8143e00 msgr2=0x7fe1b813e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1afe00640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1b8143e00 0x7fe1b813e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1afe00640 1 -- 172.21.15.82:0/1558723465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1b8070820 msgr2=0x7fe1b81f4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1afe00640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1b8070820 0x7fe1b81f4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1afe00640 1 -- 172.21.15.82:0/1558723465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00094d0 con 0x7fe1b80cd660 2024-03-16T17:57:52.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1afe00640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b80cd660 0x7fe1b81f42c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0000c00 tx=0x7fe1a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:52.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1af400640 1 -- 172.21.15.82:0/1558723465 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a003a930 con 0x7fe1b80cd660 2024-03-16T17:57:52.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1af400640 1 -- 172.21.15.82:0/1558723465 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a003aa90 con 0x7fe1b80cd660 2024-03-16T17:57:52.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1af400640 1 -- 172.21.15.82:0/1558723465 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a003ae40 con 0x7fe1b80cd660 2024-03-16T17:57:52.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1bd000640 1 -- 172.21.15.82:0/1558723465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b813c720 con 0x7fe1b80cd660 2024-03-16T17:57:52.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.007+0000 7fe1bd000640 1 -- 172.21.15.82:0/1558723465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b813cce0 con 0x7fe1b80cd660 2024-03-16T17:57:52.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.008+0000 7fe1bd000640 1 -- 172.21.15.82:0/1558723465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b80b4d80 con 0x7fe1b80cd660 2024-03-16T17:57:52.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.009+0000 7fe1af400640 1 -- 172.21.15.82:0/1558723465 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe1a002f050 con 0x7fe1b80cd660 2024-03-16T17:57:52.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.010+0000 7fe1af400640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe19406b3b0 0x7fe19406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.010+0000 7fe1af400640 1 -- 172.21.15.82:0/1558723465 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1a00abaa0 con 0x7fe1b80cd660 2024-03-16T17:57:52.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.010+0000 7fe1b7400640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe19406b3b0 0x7fe19406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.011+0000 7fe1b7400640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe19406b3b0 0x7fe19406d870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8000f60 tx=0x7fe1a8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:52.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.015+0000 7fe1af400640 1 -- 172.21.15.82:0/1558723465 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe1a0076660 con 0x7fe1b80cd660 2024-03-16T17:57:52.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.232+0000 7fe1bd000640 1 -- 172.21.15.82:0/1558723465 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe1b813ced0 con 0x7fe1b80cd660 2024-03-16T17:57:52.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.232+0000 7fe1af400640 1 -- 172.21.15.82:0/1558723465 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe1a0079d20 con 0x7fe1b80cd660 2024-03-16T17:57:52.236 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:52.236 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-16T17:57:52.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 -- 172.21.15.82:0/1558723465 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe19406b3b0 msgr2=0x7fe19406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe19406b3b0 0x7fe19406d870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8000f60 tx=0x7fe1a8009040 comp rx=0 tx=0).stop 2024-03-16T17:57:52.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 -- 172.21.15.82:0/1558723465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b80cd660 msgr2=0x7fe1b81f42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b80cd660 0x7fe1b81f42c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0000c00 tx=0x7fe1a00046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 -- 172.21.15.82:0/1558723465 shutdown_connections 2024-03-16T17:57:52.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe19406b3b0 0x7fe19406d870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1b8143e00 0x7fe1b813e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1b80cd660 0x7fe1b81f42c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 --2- 172.21.15.82:0/1558723465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1b8070820 0x7fe1b81f4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 -- 172.21.15.82:0/1558723465 >> 172.21.15.82:0/1558723465 conn(0x7fe1b806ae70 msgr2=0x7fe1b80af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:52.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 -- 172.21.15.82:0/1558723465 shutdown_connections 2024-03-16T17:57:52.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.235+0000 7fe1bd000640 1 -- 172.21.15.82:0/1558723465 wait complete. 2024-03-16T17:57:52.255 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-16T17:57:52.256 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:52.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.457+0000 7faf89800640 1 Processor -- start 2024-03-16T17:57:52.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.457+0000 7faf89800640 1 -- start start 2024-03-16T17:57:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.459+0000 7faf89800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf84144400 0x7faf840cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.459+0000 7faf89800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf8415f550 0x7faf840cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.459+0000 7faf89800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7faf8415b310 0x7faf8415b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.459+0000 7faf89800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf8415f130 con 0x7faf84144400 2024-03-16T17:57:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.459+0000 7faf89800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf8415e820 con 0x7faf8415b310 2024-03-16T17:57:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.459+0000 7faf89800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8415c080 con 0x7faf8415f550 2024-03-16T17:57:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.459+0000 7faf82a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf84144400 0x7faf840cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.459+0000 7faf82a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf84144400 0x7faf840cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35530/0 (socket says 172.21.15.82:35530) 2024-03-16T17:57:52.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.459+0000 7faf82a00640 1 -- 172.21.15.82:0/1189297565 learned_addr learned my addr 172.21.15.82:0/1189297565 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:52.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.460+0000 7faf82000640 1 --2- 172.21.15.82:0/1189297565 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf8415f550 0x7faf840cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.460+0000 7faf83400640 1 --2- 172.21.15.82:0/1189297565 >> v2:172.21.15.90:3300/0 conn(0x7faf8415b310 0x7faf8415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.460+0000 7faf82a00640 1 -- 172.21.15.82:0/1189297565 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf8415f550 msgr2=0x7faf840cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:52.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.460+0000 7faf82a00640 1 --2- 172.21.15.82:0/1189297565 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf8415f550 0x7faf840cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.460+0000 7faf82a00640 1 -- 172.21.15.82:0/1189297565 >> v2:172.21.15.90:3300/0 conn(0x7faf8415b310 msgr2=0x7faf8415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:52.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.460+0000 7faf82a00640 1 --2- 172.21.15.82:0/1189297565 >> v2:172.21.15.90:3300/0 conn(0x7faf8415b310 0x7faf8415b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.460+0000 7faf82a00640 1 -- 172.21.15.82:0/1189297565 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8415ea70 con 0x7faf84144400 2024-03-16T17:57:52.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.460+0000 7faf82000640 1 --2- 172.21.15.82:0/1189297565 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf8415f550 0x7faf840cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:52.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.460+0000 7faf83400640 1 --2- 172.21.15.82:0/1189297565 >> v2:172.21.15.90:3300/0 conn(0x7faf8415b310 0x7faf8415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:52.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.461+0000 7faf82a00640 1 --2- 172.21.15.82:0/1189297565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf84144400 0x7faf840cf990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf8406d170 tx=0x7faf7402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0fd84c7647107f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:52.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.461+0000 7faf81600640 1 -- 172.21.15.82:0/1189297565 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7402edf0 con 0x7faf84144400 2024-03-16T17:57:52.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.461+0000 7faf81600640 1 -- 172.21.15.82:0/1189297565 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf74036440 con 0x7faf84144400 2024-03-16T17:57:52.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.461+0000 7faf81600640 1 -- 172.21.15.82:0/1189297565 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf740367d0 con 0x7faf84144400 2024-03-16T17:57:52.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.462+0000 7faf89800640 1 -- 172.21.15.82:0/1189297565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf84144400 msgr2=0x7faf840cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.462+0000 7faf89800640 1 --2- 172.21.15.82:0/1189297565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf84144400 0x7faf840cf990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faf8406d170 tx=0x7faf7402e040 comp rx=0 tx=0).stop 2024-03-16T17:57:52.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.462+0000 7faf89800640 1 -- 172.21.15.82:0/1189297565 shutdown_connections 2024-03-16T17:57:52.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.462+0000 7faf89800640 1 --2- 172.21.15.82:0/1189297565 >> v2:172.21.15.90:3300/0 conn(0x7faf8415b310 0x7faf8415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.462+0000 7faf89800640 1 --2- 172.21.15.82:0/1189297565 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf8415f550 0x7faf840cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.462+0000 7faf89800640 1 --2- 172.21.15.82:0/1189297565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf84144400 0x7faf840cf990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.462+0000 7faf89800640 1 -- 172.21.15.82:0/1189297565 >> 172.21.15.82:0/1189297565 conn(0x7faf8406ae70 msgr2=0x7faf8406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:52.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.462+0000 7faf89800640 1 -- 172.21.15.82:0/1189297565 shutdown_connections 2024-03-16T17:57:52.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.462+0000 7faf89800640 1 -- 172.21.15.82:0/1189297565 wait complete. 2024-03-16T17:57:52.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.464+0000 7faf89800640 1 Processor -- start 2024-03-16T17:57:52.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.464+0000 7faf89800640 1 -- start start 2024-03-16T17:57:52.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.465+0000 7faf89800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf84144400 0x7faf841425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.465+0000 7faf89800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8415b310 0x7faf840ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.465+0000 7faf89800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf8415f550 0x7faf840ab630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.465+0000 7faf89800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf841363b0 con 0x7faf8415f550 2024-03-16T17:57:52.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.465+0000 7faf89800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf84142b00 con 0x7faf8415b310 2024-03-16T17:57:52.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.465+0000 7faf89800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf841354c0 con 0x7faf84144400 2024-03-16T17:57:52.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf83400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf8415f550 0x7faf840ab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf83400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf8415f550 0x7faf840ab630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35536/0 (socket says 172.21.15.82:35536) 2024-03-16T17:57:52.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf83400640 1 -- 172.21.15.82:0/4229396750 learned_addr learned my addr 172.21.15.82:0/4229396750 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:52.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf82a00640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf84144400 0x7faf841425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf82000640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8415b310 0x7faf840ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf83400640 1 -- 172.21.15.82:0/4229396750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf84144400 msgr2=0x7faf841425c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf83400640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf84144400 0x7faf841425c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf83400640 1 -- 172.21.15.82:0/4229396750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8415b310 msgr2=0x7faf840ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf83400640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8415b310 0x7faf840ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf83400640 1 -- 172.21.15.82:0/4229396750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7400b770 con 0x7faf8415f550 2024-03-16T17:57:52.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf82000640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8415b310 0x7faf840ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:52.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.466+0000 7faf83400640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf8415f550 0x7faf840ab630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00afd0 tx=0x7faf7c00ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:52.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.467+0000 7faf7aa00640 1 -- 172.21.15.82:0/4229396750 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c004440 con 0x7faf8415f550 2024-03-16T17:57:52.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.467+0000 7faf7aa00640 1 -- 172.21.15.82:0/4229396750 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c003610 con 0x7faf8415f550 2024-03-16T17:57:52.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.467+0000 7faf7aa00640 1 -- 172.21.15.82:0/4229396750 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0039c0 con 0x7faf8415f550 2024-03-16T17:57:52.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.467+0000 7faf89800640 1 -- 172.21.15.82:0/4229396750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf841357a0 con 0x7faf8415f550 2024-03-16T17:57:52.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.467+0000 7faf89800640 1 -- 172.21.15.82:0/4229396750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf84135d60 con 0x7faf8415f550 2024-03-16T17:57:52.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.468+0000 7faf89800640 1 -- 172.21.15.82:0/4229396750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf84067f30 con 0x7faf8415f550 2024-03-16T17:57:52.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.469+0000 7faf7aa00640 1 -- 172.21.15.82:0/4229396750 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7faf7c003b20 con 0x7faf8415f550 2024-03-16T17:57:52.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.470+0000 7faf7aa00640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf6006b3b0 0x7faf6006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.470+0000 7faf7aa00640 1 -- 172.21.15.82:0/4229396750 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf7c089ff0 con 0x7faf8415f550 2024-03-16T17:57:52.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.470+0000 7faf82a00640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf6006b3b0 0x7faf6006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.471+0000 7faf82a00640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf6006b3b0 0x7faf6006d870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf74038000 tx=0x7faf74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:52.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.474+0000 7faf7aa00640 1 -- 172.21.15.82:0/4229396750 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf7c054b20 con 0x7faf8415f550 2024-03-16T17:57:52.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.673+0000 7faf89800640 1 -- 172.21.15.82:0/4229396750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf841364f0 con 0x7faf8415f550 2024-03-16T17:57:52.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.676+0000 7faf7aa00640 1 -- 172.21.15.82:0/4229396750 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7faf7c015020 con 0x7faf8415f550 2024-03-16T17:57:52.680 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:52.680 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 -- 172.21.15.82:0/4229396750 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf6006b3b0 msgr2=0x7faf6006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf6006b3b0 0x7faf6006d870 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf74038000 tx=0x7faf74008040 comp rx=0 tx=0).stop 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 -- 172.21.15.82:0/4229396750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf8415f550 msgr2=0x7faf840ab630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf8415f550 0x7faf840ab630 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00afd0 tx=0x7faf7c00ca20 comp rx=0 tx=0).stop 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 -- 172.21.15.82:0/4229396750 shutdown_connections 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf6006b3b0 0x7faf6006d870 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf8415f550 0x7faf840ab630 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf8415b310 0x7faf840ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 --2- 172.21.15.82:0/4229396750 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf84144400 0x7faf841425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 -- 172.21.15.82:0/4229396750 >> 172.21.15.82:0/4229396750 conn(0x7faf8406ae70 msgr2=0x7faf8415dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 -- 172.21.15.82:0/4229396750 shutdown_connections 2024-03-16T17:57:52.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.679+0000 7faf89800640 1 -- 172.21.15.82:0/4229396750 wait complete. 2024-03-16T17:57:52.702 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:57:52.702 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:57:52.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.908+0000 7f74f1e00640 1 Processor -- start 2024-03-16T17:57:52.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.909+0000 7f74f1e00640 1 -- start start 2024-03-16T17:57:52.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.910+0000 7f74f1e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec1469c0 0x7f74ec162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.910+0000 7f74f1e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f74ec0cddc0 0x7f74ec0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.910+0000 7f74f1e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec146da0 con 0x7f74ec147060 2024-03-16T17:57:52.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.910+0000 7f74f1e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ec0ce7b0 con 0x7f74ec0cddc0 2024-03-16T17:57:52.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.910+0000 7f74f1e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec1628a0 con 0x7f74ec1469c0 2024-03-16T17:57:52.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.911+0000 7f74eaa00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec1469c0 0x7f74ec162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.911+0000 7f74eaa00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec1469c0 0x7f74ec162360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:60484/0 (socket says 172.21.15.82:60484) 2024-03-16T17:57:52.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.911+0000 7f74eaa00640 1 -- 172.21.15.82:0/1608670891 learned_addr learned my addr 172.21.15.82:0/1608670891 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:52.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.911+0000 7f74eb400640 1 --2- 172.21.15.82:0/1608670891 >> v2:172.21.15.90:3300/0 conn(0x7f74ec0cddc0 0x7f74ec0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.911+0000 7f74eaa00640 1 -- 172.21.15.82:0/1608670891 >> v2:172.21.15.90:3300/0 conn(0x7f74ec0cddc0 msgr2=0x7f74ec0ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:52.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.911+0000 7f74eaa00640 1 --2- 172.21.15.82:0/1608670891 >> v2:172.21.15.90:3300/0 conn(0x7f74ec0cddc0 0x7f74ec0ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.911+0000 7f74eaa00640 1 -- 172.21.15.82:0/1608670891 >> v1:172.21.15.90:6789/0 conn(0x7f74ec147060 legacy=0x7f74ec06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.911+0000 7f74eaa00640 1 -- 172.21.15.82:0/1608670891 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec06d490 con 0x7f74ec1469c0 2024-03-16T17:57:52.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.913+0000 7f74eaa00640 1 --2- 172.21.15.82:0/1608670891 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec1469c0 0x7f74ec162360 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009870 tx=0x7f74dc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3025c2f73debfa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:52.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.913+0000 7f74ea000640 1 -- 172.21.15.82:0/1608670891 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc030840 con 0x7f74ec1469c0 2024-03-16T17:57:52.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.913+0000 7f74ea000640 1 -- 172.21.15.82:0/1608670891 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc0309a0 con 0x7f74ec1469c0 2024-03-16T17:57:52.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.913+0000 7f74ea000640 1 -- 172.21.15.82:0/1608670891 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc030d30 con 0x7f74ec1469c0 2024-03-16T17:57:52.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.914+0000 7f74f1e00640 1 -- 172.21.15.82:0/1608670891 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec1469c0 msgr2=0x7f74ec162360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.914+0000 7f74f1e00640 1 --2- 172.21.15.82:0/1608670891 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec1469c0 0x7f74ec162360 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009870 tx=0x7f74dc02b980 comp rx=0 tx=0).stop 2024-03-16T17:57:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.914+0000 7f74f1e00640 1 -- 172.21.15.82:0/1608670891 shutdown_connections 2024-03-16T17:57:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.914+0000 7f74f1e00640 1 --2- 172.21.15.82:0/1608670891 >> v2:172.21.15.90:3300/0 conn(0x7f74ec0cddc0 0x7f74ec0ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.914+0000 7f74f1e00640 1 --2- 172.21.15.82:0/1608670891 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec1469c0 0x7f74ec162360 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.914+0000 7f74f1e00640 1 -- 172.21.15.82:0/1608670891 >> 172.21.15.82:0/1608670891 conn(0x7f74ec0b1640 msgr2=0x7f74ec0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.914+0000 7f74f1e00640 1 -- 172.21.15.82:0/1608670891 shutdown_connections 2024-03-16T17:57:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.914+0000 7f74f1e00640 1 -- 172.21.15.82:0/1608670891 wait complete. 2024-03-16T17:57:52.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.916+0000 7f74f1e00640 1 Processor -- start 2024-03-16T17:57:52.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.916+0000 7f74f1e00640 1 -- start start 2024-03-16T17:57:52.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.917+0000 7f74f1e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74ec0cddc0 0x7f74ec15a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.917+0000 7f74f1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1469c0 0x7f74ec1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.917+0000 7f74f1e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec147060 0x7f74ec160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.917+0000 7f74f1e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec15f570 con 0x7f74ec1469c0 2024-03-16T17:57:52.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.917+0000 7f74f1e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec15a690 con 0x7f74ec0cddc0 2024-03-16T17:57:52.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.917+0000 7f74f1e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec15e930 con 0x7f74ec147060 2024-03-16T17:57:52.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.918+0000 7f74e2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1469c0 0x7f74ec1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.918+0000 7f74e2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1469c0 0x7f74ec1606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35542/0 (socket says 172.21.15.82:35542) 2024-03-16T17:57:52.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.918+0000 7f74e2000640 1 -- 172.21.15.82:0/738246805 learned_addr learned my addr 172.21.15.82:0/738246805 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:52.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.918+0000 7f74eaa00640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74ec0cddc0 0x7f74ec15a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.918+0000 7f74eb400640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec147060 0x7f74ec160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.918+0000 7f74e2000640 1 -- 172.21.15.82:0/738246805 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec147060 msgr2=0x7f74ec160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.918+0000 7f74e2000640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec147060 0x7f74ec160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.918+0000 7f74e2000640 1 -- 172.21.15.82:0/738246805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74ec0cddc0 msgr2=0x7f74ec15a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:52.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.918+0000 7f74e2000640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74ec0cddc0 0x7f74ec15a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:52.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.918+0000 7f74e2000640 1 -- 172.21.15.82:0/738246805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc0094d0 con 0x7f74ec1469c0 2024-03-16T17:57:52.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.919+0000 7f74e2000640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1469c0 0x7f74ec1606a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f74d400c440 tx=0x7f74d400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:52.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.919+0000 7f74e3e00640 1 -- 172.21.15.82:0/738246805 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d40052d0 con 0x7f74ec1469c0 2024-03-16T17:57:52.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.919+0000 7f74e3e00640 1 -- 172.21.15.82:0/738246805 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d4005430 con 0x7f74ec1469c0 2024-03-16T17:57:52.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.919+0000 7f74e3e00640 1 -- 172.21.15.82:0/738246805 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4007260 con 0x7f74ec1469c0 2024-03-16T17:57:52.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.919+0000 7f74f1e00640 1 -- 172.21.15.82:0/738246805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec15ebd0 con 0x7f74ec1469c0 2024-03-16T17:57:52.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.919+0000 7f74f1e00640 1 -- 172.21.15.82:0/738246805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec15f190 con 0x7f74ec1469c0 2024-03-16T17:57:52.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.920+0000 7f74f1e00640 1 -- 172.21.15.82:0/738246805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec067f30 con 0x7f74ec1469c0 2024-03-16T17:57:52.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.921+0000 7f74e3e00640 1 -- 172.21.15.82:0/738246805 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f74d40073c0 con 0x7f74ec1469c0 2024-03-16T17:57:52.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.923+0000 7f74e3e00640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f74c806b3b0 0x7f74c806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:52.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.923+0000 7f74e3e00640 1 -- 172.21.15.82:0/738246805 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74d4088ec0 con 0x7f74ec1469c0 2024-03-16T17:57:52.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.923+0000 7f74eaa00640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f74c806b3b0 0x7f74c806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:52.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.924+0000 7f74eaa00640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f74c806b3b0 0x7f74c806d870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f74dc036000 tx=0x7f74dc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:52.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:52.927+0000 7f74e3e00640 1 -- 172.21.15.82:0/738246805 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f74d4053bc0 con 0x7f74ec1469c0 2024-03-16T17:57:53.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.129+0000 7f74f1e00640 1 -- 172.21.15.82:0/738246805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74ec0ca560 con 0x7f74ec1469c0 2024-03-16T17:57:53.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.130+0000 7f74e3e00640 1 -- 172.21.15.82:0/738246805 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74d4057280 con 0x7f74ec1469c0 2024-03-16T17:57:53.133 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 -- 172.21.15.82:0/738246805 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f74c806b3b0 msgr2=0x7f74c806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f74c806b3b0 0x7f74c806d870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f74dc036000 tx=0x7f74dc008030 comp rx=0 tx=0).stop 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 -- 172.21.15.82:0/738246805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1469c0 msgr2=0x7f74ec1606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1469c0 0x7f74ec1606a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f74d400c440 tx=0x7f74d400c900 comp rx=0 tx=0).stop 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 -- 172.21.15.82:0/738246805 shutdown_connections 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f74c806b3b0 0x7f74c806d870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f74ec147060 0x7f74ec160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74ec1469c0 0x7f74ec1606a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 --2- 172.21.15.82:0/738246805 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74ec0cddc0 0x7f74ec15a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 -- 172.21.15.82:0/738246805 >> 172.21.15.82:0/738246805 conn(0x7f74ec0b1640 msgr2=0x7f74ec071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 -- 172.21.15.82:0/738246805 shutdown_connections 2024-03-16T17:57:53.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.133+0000 7f74f1e00640 1 -- 172.21.15.82:0/738246805 wait complete. 2024-03-16T17:57:53.156 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:57:53.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.365+0000 7fc87de00640 1 Processor -- start 2024-03-16T17:57:53.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.365+0000 7fc87de00640 1 -- start start 2024-03-16T17:57:53.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.366+0000 7fc87de00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 0x7fc878070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.366+0000 7fc87de00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fc8780714f0 0x7fc87815e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.367+0000 7fc87de00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8780ce8f0 con 0x7fc87815eca0 2024-03-16T17:57:53.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.367+0000 7fc87de00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87815f2d0 con 0x7fc8780714f0 2024-03-16T17:57:53.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.367+0000 7fc87de00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87815f5b0 con 0x7fc878144330 2024-03-16T17:57:53.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.367+0000 7fc877400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc87815eca0 0x7fc8780cc380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56312/0 (socket says 172.21.15.82:56312) 2024-03-16T17:57:53.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.367+0000 7fc877400640 1 -- 172.21.15.82:0/840283702 learned_addr learned my addr 172.21.15.82:0/840283702 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:53.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.367+0000 7fc876a00640 1 --2- 172.21.15.82:0/840283702 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 0x7fc878070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.367+0000 7fc86fe00640 1 --2- 172.21.15.82:0/840283702 >> v2:172.21.15.90:3300/0 conn(0x7fc8780714f0 0x7fc87815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.367+0000 7fc876000640 1 -- 172.21.15.82:0/840283702 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882202090 0 0) 0x7fc8780ce8f0 con 0x7fc87815eca0 2024-03-16T17:57:53.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.367+0000 7fc876000640 1 -- 172.21.15.82:0/840283702 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc858003520 con 0x7fc87815eca0 2024-03-16T17:57:53.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.368+0000 7fc876a00640 1 -- 172.21.15.82:0/840283702 >> v2:172.21.15.90:3300/0 conn(0x7fc8780714f0 msgr2=0x7fc87815e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:53.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.368+0000 7fc876a00640 1 --2- 172.21.15.82:0/840283702 >> v2:172.21.15.90:3300/0 conn(0x7fc8780714f0 0x7fc87815e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.368+0000 7fc876a00640 1 -- 172.21.15.82:0/840283702 >> v1:172.21.15.82:6789/0 conn(0x7fc87815eca0 legacy=0x7fc8780cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.368+0000 7fc876000640 1 -- 172.21.15.82:0/840283702 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3449608849 0 0) 0x7fc858003520 con 0x7fc87815eca0 2024-03-16T17:57:53.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.368+0000 7fc876a00640 1 -- 172.21.15.82:0/840283702 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780cacb0 con 0x7fc878144330 2024-03-16T17:57:53.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.369+0000 7fc876a00640 1 --2- 172.21.15.82:0/840283702 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 0x7fc878070fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc8600097a0 tx=0x7fc86002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7635fb65cb419efc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:53.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.369+0000 7fc876000640 1 -- 172.21.15.82:0/840283702 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8700019f0 con 0x7fc878144330 2024-03-16T17:57:53.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.369+0000 7fc876000640 1 -- 172.21.15.82:0/840283702 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86002e050 con 0x7fc878144330 2024-03-16T17:57:53.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.369+0000 7fc876000640 1 -- 172.21.15.82:0/840283702 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860030730 con 0x7fc878144330 2024-03-16T17:57:53.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.370+0000 7fc87de00640 1 -- 172.21.15.82:0/840283702 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 msgr2=0x7fc878070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.370+0000 7fc87de00640 1 --2- 172.21.15.82:0/840283702 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 0x7fc878070fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc8600097a0 tx=0x7fc86002b790 comp rx=0 tx=0).stop 2024-03-16T17:57:53.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.370+0000 7fc87de00640 1 -- 172.21.15.82:0/840283702 shutdown_connections 2024-03-16T17:57:53.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.370+0000 7fc87de00640 1 --2- 172.21.15.82:0/840283702 >> v2:172.21.15.90:3300/0 conn(0x7fc8780714f0 0x7fc87815e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.370+0000 7fc87de00640 1 --2- 172.21.15.82:0/840283702 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 0x7fc878070fb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.370+0000 7fc87de00640 1 -- 172.21.15.82:0/840283702 >> 172.21.15.82:0/840283702 conn(0x7fc8780b1640 msgr2=0x7fc8780b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:53.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.370+0000 7fc87de00640 1 -- 172.21.15.82:0/840283702 shutdown_connections 2024-03-16T17:57:53.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.370+0000 7fc87de00640 1 -- 172.21.15.82:0/840283702 wait complete. 2024-03-16T17:57:53.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.372+0000 7fc87de00640 1 Processor -- start 2024-03-16T17:57:53.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.372+0000 7fc87de00640 1 -- start start 2024-03-16T17:57:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc87de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780714f0 0x7fc8781f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc87de00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 0x7fc8780afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc87de00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc87815eca0 0x7fc8780c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc87de00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8781f2b00 con 0x7fc8780714f0 2024-03-16T17:57:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc87de00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780c96f0 con 0x7fc87815eca0 2024-03-16T17:57:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc87de00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780c9410 con 0x7fc878144330 2024-03-16T17:57:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc876a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780714f0 0x7fc8781f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc876a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780714f0 0x7fc8781f25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35556/0 (socket says 172.21.15.82:35556) 2024-03-16T17:57:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc876a00640 1 -- 172.21.15.82:0/1351497286 learned_addr learned my addr 172.21.15.82:0/1351497286 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:53.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc86fe00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 0x7fc8780afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc876a00640 1 -- 172.21.15.82:0/1351497286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 msgr2=0x7fc8780afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc876a00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 0x7fc8780afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.373+0000 7fc876a00640 1 -- 172.21.15.82:0/1351497286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc87815eca0 msgr2=0x7fc8780c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.374+0000 7fc877400640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc87815eca0 0x7fc8780c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.374+0000 7fc876a00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc87815eca0 0x7fc8780c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.374+0000 7fc876a00640 1 -- 172.21.15.82:0/1351497286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc860009400 con 0x7fc8780714f0 2024-03-16T17:57:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.374+0000 7fc876a00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780714f0 0x7fc8781f25c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc860000c00 tx=0x7fc8600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.374+0000 7fc86f400640 1 -- 172.21.15.82:0/1351497286 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86003b810 con 0x7fc8780714f0 2024-03-16T17:57:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.374+0000 7fc86f400640 1 -- 172.21.15.82:0/1351497286 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86003b970 con 0x7fc8780714f0 2024-03-16T17:57:53.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.374+0000 7fc86f400640 1 -- 172.21.15.82:0/1351497286 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86003bd20 con 0x7fc8780714f0 2024-03-16T17:57:53.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.374+0000 7fc87de00640 1 -- 172.21.15.82:0/1351497286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8780c6430 con 0x7fc8780714f0 2024-03-16T17:57:53.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.374+0000 7fc87de00640 1 -- 172.21.15.82:0/1351497286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8780c69f0 con 0x7fc8780714f0 2024-03-16T17:57:53.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.376+0000 7fc87de00640 1 -- 172.21.15.82:0/1351497286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc878067f30 con 0x7fc8780714f0 2024-03-16T17:57:53.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.376+0000 7fc86f400640 1 -- 172.21.15.82:0/1351497286 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fc860039040 con 0x7fc8780714f0 2024-03-16T17:57:53.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.378+0000 7fc86f400640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc85406b3b0 0x7fc85406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.378+0000 7fc86f400640 1 -- 172.21.15.82:0/1351497286 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8600ab280 con 0x7fc8780714f0 2024-03-16T17:57:53.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.378+0000 7fc86fe00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc85406b3b0 0x7fc85406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.378+0000 7fc86fe00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc85406b3b0 0x7fc85406d870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc8780af500 tx=0x7fc868008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:53.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.382+0000 7fc86f400640 1 -- 172.21.15.82:0/1351497286 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc860075f70 con 0x7fc8780714f0 2024-03-16T17:57:53.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.578+0000 7fc87de00640 1 -- 172.21.15.82:0/1351497286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8780ca540 con 0x7fc8780714f0 2024-03-16T17:57:53.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.579+0000 7fc86f400640 1 -- 172.21.15.82:0/1351497286 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc860079630 con 0x7fc8780714f0 2024-03-16T17:57:53.583 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:57:53.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 -- 172.21.15.82:0/1351497286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc85406b3b0 msgr2=0x7fc85406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc85406b3b0 0x7fc85406d870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc8780af500 tx=0x7fc868008040 comp rx=0 tx=0).stop 2024-03-16T17:57:53.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 -- 172.21.15.82:0/1351497286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780714f0 msgr2=0x7fc8781f25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780714f0 0x7fc8781f25c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc860000c00 tx=0x7fc8600046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 -- 172.21.15.82:0/1351497286 shutdown_connections 2024-03-16T17:57:53.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc85406b3b0 0x7fc85406d870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc87815eca0 0x7fc8780c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc878144330 0x7fc8780afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 --2- 172.21.15.82:0/1351497286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780714f0 0x7fc8781f25c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 -- 172.21.15.82:0/1351497286 >> 172.21.15.82:0/1351497286 conn(0x7fc8780b1640 msgr2=0x7fc8780cda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:53.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 -- 172.21.15.82:0/1351497286 shutdown_connections 2024-03-16T17:57:53.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.582+0000 7fc87de00640 1 -- 172.21.15.82:0/1351497286 wait complete. 2024-03-16T17:57:53.604 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:57:53.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.812+0000 7fec47200640 1 Processor -- start 2024-03-16T17:57:53.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.812+0000 7fec47200640 1 -- start start 2024-03-16T17:57:53.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.814+0000 7fec47200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fec40147060 0x7fec4006e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.814+0000 7fec47200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec40146da0 con 0x7fec4006ea70 2024-03-16T17:57:53.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.814+0000 7fec47200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4006d490 con 0x7fec401469c0 2024-03-16T17:57:53.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.814+0000 7fec47200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4006d770 con 0x7fec40147060 2024-03-16T17:57:53.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.814+0000 7fec45200640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fec4006ea70 0x7fec400cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56328/0 (socket says 172.21.15.82:56328) 2024-03-16T17:57:53.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.814+0000 7fec45200640 1 -- 172.21.15.82:0/2233920505 learned_addr learned my addr 172.21.15.82:0/2233920505 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:53.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.814+0000 7fec3f400640 1 --2- 172.21.15.82:0/2233920505 >> v2:172.21.15.90:3300/0 conn(0x7fec40147060 0x7fec4006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046292558 0 0) 0x7fec40146da0 con 0x7fec4006ea70 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec24003520 con 0x7fec4006ea70 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2080978039 0 0) 0x7fec24003520 con 0x7fec4006ea70 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 >> v2:172.21.15.90:3300/0 conn(0x7fec40147060 msgr2=0x7fec4006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 --2- 172.21.15.82:0/2233920505 >> v2:172.21.15.90:3300/0 conn(0x7fec40147060 0x7fec4006e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 >> v1:172.21.15.90:6789/0 conn(0x7fec401469c0 legacy=0x7fec40162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4006da50 con 0x7fec4006ea70 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fec380019f0 con 0x7fec4006ea70 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 >> v1:172.21.15.82:6789/0 conn(0x7fec4006ea70 legacy=0x7fec400cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 --2- 172.21.15.82:0/2233920505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec240039c0 0x7fec24003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 --2- 172.21.15.82:0/2233920505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec240047c0 0x7fec24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 --2- 172.21.15.82:0/2233920505 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec24005180 0x7fec2400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4006d770 con 0x7fec240047c0 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2400c080 con 0x7fec240039c0 2024-03-16T17:57:53.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2400c240 con 0x7fec24005180 2024-03-16T17:57:53.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.815+0000 7fec3f400640 1 --2- 172.21.15.82:0/2233920505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec240047c0 0x7fec24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.816+0000 7fec3fe00640 1 --2- 172.21.15.82:0/2233920505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec240039c0 0x7fec24003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.816+0000 7fec3f400640 1 -- 172.21.15.82:0/2233920505 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec24005180 msgr2=0x7fec2400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:53.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.816+0000 7fec3f400640 1 --2- 172.21.15.82:0/2233920505 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec24005180 0x7fec2400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.816+0000 7fec3f400640 1 -- 172.21.15.82:0/2233920505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec240039c0 msgr2=0x7fec24003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:53.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.816+0000 7fec3f400640 1 --2- 172.21.15.82:0/2233920505 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec240039c0 0x7fec24003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.816+0000 7fec3f400640 1 -- 172.21.15.82:0/2233920505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec24003520 con 0x7fec240047c0 2024-03-16T17:57:53.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.816+0000 7fec45200640 1 -- 172.21.15.82:0/2233920505 reap_dead start 2024-03-16T17:57:53.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.817+0000 7fec3f400640 1 --2- 172.21.15.82:0/2233920505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec240047c0 0x7fec24004c40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009ac0 tx=0x7fec240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6e87163315f9f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:53.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.817+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c02c6d0 con 0x7fec240047c0 2024-03-16T17:57:53.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.817+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2c02c830 con 0x7fec240047c0 2024-03-16T17:57:53.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.817+0000 7fec3ea00640 1 -- 172.21.15.82:0/2233920505 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2c02cbc0 con 0x7fec240047c0 2024-03-16T17:57:53.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.818+0000 7fec47200640 1 -- 172.21.15.82:0/2233920505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec240047c0 msgr2=0x7fec24004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.818+0000 7fec47200640 1 --2- 172.21.15.82:0/2233920505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec240047c0 0x7fec24004c40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009ac0 tx=0x7fec240057f0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.818+0000 7fec47200640 1 -- 172.21.15.82:0/2233920505 shutdown_connections 2024-03-16T17:57:53.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.818+0000 7fec47200640 1 --2- 172.21.15.82:0/2233920505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec240047c0 0x7fec24004c40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.818+0000 7fec47200640 1 -- 172.21.15.82:0/2233920505 >> 172.21.15.82:0/2233920505 conn(0x7fec400b1640 msgr2=0x7fec400b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:53.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.818+0000 7fec47200640 1 -- 172.21.15.82:0/2233920505 shutdown_connections 2024-03-16T17:57:53.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.818+0000 7fec47200640 1 -- 172.21.15.82:0/2233920505 wait complete. 2024-03-16T17:57:53.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.820+0000 7fec47200640 1 Processor -- start 2024-03-16T17:57:53.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.820+0000 7fec47200640 1 -- start start 2024-03-16T17:57:53.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.821+0000 7fec47200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec240047c0 0x7fec400b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.821+0000 7fec47200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec4015d260 0x7fec400b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.821+0000 7fec47200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec4015d6a0 0x7fec40160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.821+0000 7fec47200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b2b70 con 0x7fec4015d6a0 2024-03-16T17:57:53.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.821+0000 7fec47200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b2e50 con 0x7fec240047c0 2024-03-16T17:57:53.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.821+0000 7fec47200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b3130 con 0x7fec4015d260 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.821+0000 7fec45200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec4015d6a0 0x7fec40160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.821+0000 7fec45200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec4015d6a0 0x7fec40160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35572/0 (socket says 172.21.15.82:35572) 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.821+0000 7fec45200640 1 -- 172.21.15.82:0/917846733 learned_addr learned my addr 172.21.15.82:0/917846733 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.821+0000 7fec3f400640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec4015d260 0x7fec400b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec3fe00640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec240047c0 0x7fec400b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec45200640 1 -- 172.21.15.82:0/917846733 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec4015d260 msgr2=0x7fec400b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec45200640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec4015d260 0x7fec400b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec45200640 1 -- 172.21.15.82:0/917846733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec240047c0 msgr2=0x7fec400b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec45200640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec240047c0 0x7fec400b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec45200640 1 -- 172.21.15.82:0/917846733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400b3410 con 0x7fec4015d6a0 2024-03-16T17:57:53.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec45200640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec4015d6a0 0x7fec40160500 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec400cda20 tx=0x7fec3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:53.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec37e00640 1 -- 172.21.15.82:0/917846733 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38007c50 con 0x7fec4015d6a0 2024-03-16T17:57:53.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec37e00640 1 -- 172.21.15.82:0/917846733 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3800a610 con 0x7fec4015d6a0 2024-03-16T17:57:53.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec37e00640 1 -- 172.21.15.82:0/917846733 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3800aa00 con 0x7fec4015d6a0 2024-03-16T17:57:53.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec47200640 1 -- 172.21.15.82:0/917846733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec400b36f0 con 0x7fec4015d6a0 2024-03-16T17:57:53.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.822+0000 7fec47200640 1 -- 172.21.15.82:0/917846733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec400b55a0 con 0x7fec4015d6a0 2024-03-16T17:57:53.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.823+0000 7fec47200640 1 -- 172.21.15.82:0/917846733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4006d100 con 0x7fec4015d6a0 2024-03-16T17:57:53.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.824+0000 7fec37e00640 1 -- 172.21.15.82:0/917846733 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fec3800ab60 con 0x7fec4015d6a0 2024-03-16T17:57:53.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.825+0000 7fec37e00640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fec2006b3e0 0x7fec2006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:53.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.825+0000 7fec37e00640 1 -- 172.21.15.82:0/917846733 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec380898f0 con 0x7fec4015d6a0 2024-03-16T17:57:53.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.826+0000 7fec3fe00640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fec2006b3e0 0x7fec2006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:53.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.826+0000 7fec3fe00640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fec2006b3e0 0x7fec2006d8a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fec400504c0 tx=0x7fec30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:53.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:53.830+0000 7fec37e00640 1 -- 172.21.15.82:0/917846733 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fec380544b0 con 0x7fec4015d6a0 2024-03-16T17:57:54.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.031+0000 7fec47200640 1 -- 172.21.15.82:0/917846733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec400b38e0 con 0x7fec4015d6a0 2024-03-16T17:57:54.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.032+0000 7fec37e00640 1 -- 172.21.15.82:0/917846733 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec38057b70 con 0x7fec4015d6a0 2024-03-16T17:57:54.036 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:57:54.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.034+0000 7fec47200640 1 -- 172.21.15.82:0/917846733 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fec2006b3e0 msgr2=0x7fec2006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.034+0000 7fec47200640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fec2006b3e0 0x7fec2006d8a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fec400504c0 tx=0x7fec30009040 comp rx=0 tx=0).stop 2024-03-16T17:57:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.034+0000 7fec47200640 1 -- 172.21.15.82:0/917846733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec4015d6a0 msgr2=0x7fec40160500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.034+0000 7fec47200640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec4015d6a0 0x7fec40160500 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fec400cda20 tx=0x7fec3800ccb0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.035+0000 7fec47200640 1 -- 172.21.15.82:0/917846733 shutdown_connections 2024-03-16T17:57:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.035+0000 7fec47200640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fec2006b3e0 0x7fec2006d8a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.035+0000 7fec47200640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec4015d6a0 0x7fec40160500 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.035+0000 7fec47200640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fec4015d260 0x7fec400b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.035+0000 7fec47200640 1 --2- 172.21.15.82:0/917846733 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec240047c0 0x7fec400b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.035+0000 7fec47200640 1 -- 172.21.15.82:0/917846733 >> 172.21.15.82:0/917846733 conn(0x7fec400b1640 msgr2=0x7fec40161e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.035+0000 7fec47200640 1 -- 172.21.15.82:0/917846733 shutdown_connections 2024-03-16T17:57:54.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.035+0000 7fec47200640 1 -- 172.21.15.82:0/917846733 wait complete. 2024-03-16T17:57:54.057 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:57:54.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.252+0000 7fc2b0800640 1 Processor -- start 2024-03-16T17:57:54.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.252+0000 7fc2b0800640 1 -- start start 2024-03-16T17:57:54.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.253+0000 7fc2b0800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc2a8144400 0x7fc2a80cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.253+0000 7fc2b0800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a815e820 con 0x7fc2a815f550 2024-03-16T17:57:54.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.253+0000 7fc2b0800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a815c080 con 0x7fc2a815ba20 2024-03-16T17:57:54.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.253+0000 7fc2b0800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a815f130 con 0x7fc2a8144400 2024-03-16T17:57:54.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.254+0000 7fc2ade00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc2a8144400 0x7fc2a80cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.254+0000 7fc2ad400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc2a815f550 0x7fc2a815b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56334/0 (socket says 172.21.15.82:56334) 2024-03-16T17:57:54.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.254+0000 7fc2ad400640 1 -- 172.21.15.82:0/3260419220 learned_addr learned my addr 172.21.15.82:0/3260419220 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:54.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.254+0000 7fc2ade00640 1 -- 172.21.15.82:0/3260419220 >> v1:172.21.15.90:6789/0 conn(0x7fc2a815ba20 legacy=0x7fc2a806d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:54.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.254+0000 7fc2ade00640 1 -- 172.21.15.82:0/3260419220 >> v1:172.21.15.82:6789/0 conn(0x7fc2a815f550 legacy=0x7fc2a815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.254+0000 7fc2ade00640 1 -- 172.21.15.82:0/3260419220 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a815ea70 con 0x7fc2a8144400 2024-03-16T17:57:54.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.254+0000 7fc2aca00640 1 -- 172.21.15.82:0/3260419220 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108258767 0 0) 0x7fc2a815e820 con 0x7fc2a815f550 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.255+0000 7fc2ade00640 1 --2- 172.21.15.82:0/3260419220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a8144400 0x7fc2a80cf990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0097a0 tx=0x7fc29c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f4206a9b2a94ad server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.255+0000 7fc2aca00640 1 -- 172.21.15.82:0/3260419220 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c02e070 con 0x7fc2a8144400 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.256+0000 7fc2aca00640 1 -- 172.21.15.82:0/3260419220 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c0315a0 con 0x7fc2a8144400 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.256+0000 7fc2aca00640 1 -- 172.21.15.82:0/3260419220 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c031930 con 0x7fc2a8144400 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.256+0000 7fc2b0800640 1 -- 172.21.15.82:0/3260419220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a8144400 msgr2=0x7fc2a80cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.256+0000 7fc2b0800640 1 --2- 172.21.15.82:0/3260419220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a8144400 0x7fc2a80cf990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0097a0 tx=0x7fc29c02b840 comp rx=0 tx=0).stop 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.256+0000 7fc2b0800640 1 -- 172.21.15.82:0/3260419220 shutdown_connections 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.256+0000 7fc2b0800640 1 --2- 172.21.15.82:0/3260419220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a8144400 0x7fc2a80cf990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.256+0000 7fc2b0800640 1 -- 172.21.15.82:0/3260419220 >> 172.21.15.82:0/3260419220 conn(0x7fc2a806ae70 msgr2=0x7fc2a806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.256+0000 7fc2b0800640 1 -- 172.21.15.82:0/3260419220 shutdown_connections 2024-03-16T17:57:54.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.256+0000 7fc2b0800640 1 -- 172.21.15.82:0/3260419220 wait complete. 2024-03-16T17:57:54.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.258+0000 7fc2b0800640 1 Processor -- start 2024-03-16T17:57:54.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.258+0000 7fc2b0800640 1 -- start start 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2b0800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2a8144400 0x7fc2a8142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2b0800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a815ba20 0x7fc2a80b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2b0800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2a815f550 0x7fc2a8157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2b0800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8142ec0 con 0x7fc2a815ba20 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2b0800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8158b30 con 0x7fc2a8144400 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2b0800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80b21f0 con 0x7fc2a815f550 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2ad400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a815ba20 0x7fc2a80b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2ad400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a815ba20 0x7fc2a80b1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35600/0 (socket says 172.21.15.82:35600) 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2ad400640 1 -- 172.21.15.82:0/2521438603 learned_addr learned my addr 172.21.15.82:0/2521438603 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2ade00640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2a8144400 0x7fc2a8142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.259+0000 7fc2ae800640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2a815f550 0x7fc2a8157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2ad400640 1 -- 172.21.15.82:0/2521438603 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2a815f550 msgr2=0x7fc2a8157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2ad400640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2a815f550 0x7fc2a8157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2ad400640 1 -- 172.21.15.82:0/2521438603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2a8144400 msgr2=0x7fc2a8142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2ad400640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2a8144400 0x7fc2a8142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2ad400640 1 -- 172.21.15.82:0/2521438603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c009400 con 0x7fc2a815ba20 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2ade00640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2a8144400 0x7fc2a8142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2ad400640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a815ba20 0x7fc2a80b1cb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c430 tx=0x7fc29800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2a2000640 1 -- 172.21.15.82:0/2521438603 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2980074e0 con 0x7fc2a815ba20 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2a2000640 1 -- 172.21.15.82:0/2521438603 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc298007660 con 0x7fc2a815ba20 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2b0800640 1 -- 172.21.15.82:0/2521438603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a80b2490 con 0x7fc2a815ba20 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2a2000640 1 -- 172.21.15.82:0/2521438603 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298004a20 con 0x7fc2a815ba20 2024-03-16T17:57:54.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.260+0000 7fc2b0800640 1 -- 172.21.15.82:0/2521438603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a81584e0 con 0x7fc2a815ba20 2024-03-16T17:57:54.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.261+0000 7fc2b0800640 1 -- 172.21.15.82:0/2521438603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a8067f30 con 0x7fc2a815ba20 2024-03-16T17:57:54.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.262+0000 7fc2a2000640 1 -- 172.21.15.82:0/2521438603 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fc298004b80 con 0x7fc2a815ba20 2024-03-16T17:57:54.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.264+0000 7fc2a2000640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc28c06b3e0 0x7fc28c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.264+0000 7fc2a2000640 1 -- 172.21.15.82:0/2521438603 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc298089ac0 con 0x7fc2a815ba20 2024-03-16T17:57:54.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.264+0000 7fc2ade00640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc28c06b3e0 0x7fc28c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.265+0000 7fc2ade00640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc28c06b3e0 0x7fc28c06d8a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc29c036000 tx=0x7fc29c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:54.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.268+0000 7fc2a2000640 1 -- 172.21.15.82:0/2521438603 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc298054680 con 0x7fc2a815ba20 2024-03-16T17:57:54.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.489+0000 7fc2b0800640 1 -- 172.21.15.82:0/2521438603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2a806d460 con 0x7fc2a815ba20 2024-03-16T17:57:54.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.490+0000 7fc2a2000640 1 -- 172.21.15.82:0/2521438603 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4968 (secure 0 0 0) 0x7fc298057d40 con 0x7fc2a815ba20 2024-03-16T17:57:54.494 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:54.495 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2570794110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2570794110},{"type":"v1","addr":"172.21.15.90:6835","nonce":2570794110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3135739259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3135739259},{"type":"v1","addr":"172.21.15.82:6837","nonce":3135739259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3721205060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3721205060},{"type":"v1","addr":"172.21.15.82:6835","nonce":3721205060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:57:40.797669+0000","modified":"2024-03-16T17:57:43.865338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/732041173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":732041173},{"type":"v1","addr":"172.21.15.90:6833","nonce":732041173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4267,"qdb_cluster":[4267]},"id":1}]} 2024-03-16T17:57:54.495 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-03-16T17:57:54.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.494+0000 7fc2b0800640 1 -- 172.21.15.82:0/2521438603 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc28c06b3e0 msgr2=0x7fc28c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.494+0000 7fc2b0800640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc28c06b3e0 0x7fc28c06d8a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc29c036000 tx=0x7fc29c008030 comp rx=0 tx=0).stop 2024-03-16T17:57:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.494+0000 7fc2b0800640 1 -- 172.21.15.82:0/2521438603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a815ba20 msgr2=0x7fc2a80b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.494+0000 7fc2b0800640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a815ba20 0x7fc2a80b1cb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c430 tx=0x7fc29800c8f0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.494+0000 7fc2b0800640 1 -- 172.21.15.82:0/2521438603 shutdown_connections 2024-03-16T17:57:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.494+0000 7fc2b0800640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc28c06b3e0 0x7fc28c06d8a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.494+0000 7fc2b0800640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc2a815f550 0x7fc2a8157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.494+0000 7fc2b0800640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc2a815ba20 0x7fc2a80b1cb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.494+0000 7fc2b0800640 1 --2- 172.21.15.82:0/2521438603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc2a8144400 0x7fc2a8142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.494+0000 7fc2b0800640 1 -- 172.21.15.82:0/2521438603 >> 172.21.15.82:0/2521438603 conn(0x7fc2a806ae70 msgr2=0x7fc2a806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.495+0000 7fc2b0800640 1 -- 172.21.15.82:0/2521438603 shutdown_connections 2024-03-16T17:57:54.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.495+0000 7fc2b0800640 1 -- 172.21.15.82:0/2521438603 wait complete. 2024-03-16T17:57:54.516 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:54.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.717+0000 7f83d4000640 1 Processor -- start 2024-03-16T17:57:54.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.717+0000 7f83d4000640 1 -- start start 2024-03-16T17:57:54.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.719+0000 7f83d4000640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f83cc144400 0x7f83cc0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.719+0000 7f83d4000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83cc15ba20 0x7f83cc0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.719+0000 7f83d4000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83cc15e820 con 0x7f83cc15f550 2024-03-16T17:57:54.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.719+0000 7f83d4000640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83cc15f130 con 0x7f83cc144400 2024-03-16T17:57:54.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.719+0000 7f83d4000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc15be60 con 0x7f83cc15ba20 2024-03-16T17:57:54.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.720+0000 7f83d1600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f83cc144400 0x7f83cc0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.720+0000 7f83d2000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83cc15ba20 0x7f83cc0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.720+0000 7f83d1600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f83cc144400 0x7f83cc0cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:60772/0 (socket says 172.21.15.82:60772) 2024-03-16T17:57:54.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.720+0000 7f83d1600640 1 -- 172.21.15.82:0/3702859716 learned_addr learned my addr 172.21.15.82:0/3702859716 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:54.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.720+0000 7f83d1600640 1 -- 172.21.15.82:0/3702859716 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83cc15ba20 msgr2=0x7f83cc0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:54.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.720+0000 7f83d1600640 1 --2- 172.21.15.82:0/3702859716 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83cc15ba20 0x7f83cc0cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.720+0000 7f83d1600640 1 -- 172.21.15.82:0/3702859716 >> v1:172.21.15.90:6789/0 conn(0x7f83cc15f550 legacy=0x7f83cc15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.721+0000 7f83d1600640 1 -- 172.21.15.82:0/3702859716 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc15c080 con 0x7f83cc144400 2024-03-16T17:57:54.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.722+0000 7f83d1600640 1 --2- 172.21.15.82:0/3702859716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83cc144400 0x7f83cc0cf990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83c00098a0 tx=0x7f83c002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33a1a343234532a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:54.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.722+0000 7f83cbe00640 1 -- 172.21.15.82:0/3702859716 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0031640 con 0x7f83cc144400 2024-03-16T17:57:54.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.722+0000 7f83cbe00640 1 -- 172.21.15.82:0/3702859716 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c00317a0 con 0x7f83cc144400 2024-03-16T17:57:54.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.722+0000 7f83cbe00640 1 -- 172.21.15.82:0/3702859716 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0031b30 con 0x7f83cc144400 2024-03-16T17:57:54.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.723+0000 7f83d4000640 1 -- 172.21.15.82:0/3702859716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83cc144400 msgr2=0x7f83cc0cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.723+0000 7f83d4000640 1 --2- 172.21.15.82:0/3702859716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83cc144400 0x7f83cc0cf990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f83c00098a0 tx=0x7f83c002b8f0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.723+0000 7f83d4000640 1 -- 172.21.15.82:0/3702859716 shutdown_connections 2024-03-16T17:57:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.723+0000 7f83d4000640 1 --2- 172.21.15.82:0/3702859716 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83cc15ba20 0x7f83cc0cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.723+0000 7f83d4000640 1 --2- 172.21.15.82:0/3702859716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83cc144400 0x7f83cc0cf990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.723+0000 7f83d4000640 1 -- 172.21.15.82:0/3702859716 >> 172.21.15.82:0/3702859716 conn(0x7f83cc06ae70 msgr2=0x7f83cc06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.723+0000 7f83d4000640 1 -- 172.21.15.82:0/3702859716 shutdown_connections 2024-03-16T17:57:54.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.723+0000 7f83d4000640 1 -- 172.21.15.82:0/3702859716 wait complete. 2024-03-16T17:57:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.725+0000 7f83d4000640 1 Processor -- start 2024-03-16T17:57:54.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.725+0000 7f83d4000640 1 -- start start 2024-03-16T17:57:54.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d4000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83cc144400 0x7f83cc1408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d4000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83cc15ba20 0x7f83cc157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d4000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc15f550 0x7f83cc157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d4000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc15d940 con 0x7f83cc15f550 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d4000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc15dc20 con 0x7f83cc15ba20 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d4000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc15df00 con 0x7f83cc144400 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc15f550 0x7f83cc157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc15f550 0x7f83cc157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35602/0 (socket says 172.21.15.82:35602) 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d2000640 1 -- 172.21.15.82:0/3664351552 learned_addr learned my addr 172.21.15.82:0/3664351552 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d1600640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83cc144400 0x7f83cc1408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d0c00640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83cc15ba20 0x7f83cc157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d2000640 1 -- 172.21.15.82:0/3664351552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83cc144400 msgr2=0x7f83cc1408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d2000640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83cc144400 0x7f83cc1408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d2000640 1 -- 172.21.15.82:0/3664351552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83cc15ba20 msgr2=0x7f83cc157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d2000640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83cc15ba20 0x7f83cc157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.727+0000 7f83d2000640 1 -- 172.21.15.82:0/3664351552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c00094d0 con 0x7f83cc15f550 2024-03-16T17:57:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.728+0000 7f83d2000640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc15f550 0x7f83cc157740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f83c400efc0 tx=0x7f83c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.728+0000 7f83c9600640 1 -- 172.21.15.82:0/3664351552 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4009ba0 con 0x7f83cc15f550 2024-03-16T17:57:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.728+0000 7f83c9600640 1 -- 172.21.15.82:0/3664351552 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c400f040 con 0x7f83cc15f550 2024-03-16T17:57:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.728+0000 7f83d4000640 1 -- 172.21.15.82:0/3664351552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83cc15e1e0 con 0x7f83cc15f550 2024-03-16T17:57:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.728+0000 7f83c9600640 1 -- 172.21.15.82:0/3664351552 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40146c0 con 0x7f83cc15f550 2024-03-16T17:57:54.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.728+0000 7f83d4000640 1 -- 172.21.15.82:0/3664351552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83cc1f7630 con 0x7f83cc15f550 2024-03-16T17:57:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.729+0000 7f83d4000640 1 -- 172.21.15.82:0/3664351552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83cc067f30 con 0x7f83cc15f550 2024-03-16T17:57:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.730+0000 7f83c9600640 1 -- 172.21.15.82:0/3664351552 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f83c4014820 con 0x7f83cc15f550 2024-03-16T17:57:54.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.731+0000 7f83c9600640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83ac06b3e0 0x7f83ac06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:54.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.731+0000 7f83c9600640 1 -- 172.21.15.82:0/3664351552 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83c408a710 con 0x7f83cc15f550 2024-03-16T17:57:54.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.731+0000 7f83d1600640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83ac06b3e0 0x7f83ac06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:54.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.732+0000 7f83d1600640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83ac06b3e0 0x7f83ac06d8a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83c0036000 tx=0x7f83c003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:54.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.735+0000 7f83c9600640 1 -- 172.21.15.82:0/3664351552 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f83c4055360 con 0x7f83cc15f550 2024-03-16T17:57:54.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.932+0000 7f83d4000640 1 -- 172.21.15.82:0/3664351552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83cc06d460 con 0x7f83cc15f550 2024-03-16T17:57:54.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.935+0000 7f83c9600640 1 -- 172.21.15.82:0/3664351552 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f83c4058a20 con 0x7f83cc15f550 2024-03-16T17:57:54.939 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:54.939 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:54.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 -- 172.21.15.82:0/3664351552 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83ac06b3e0 msgr2=0x7f83ac06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83ac06b3e0 0x7f83ac06d8a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83c0036000 tx=0x7f83c003c040 comp rx=0 tx=0).stop 2024-03-16T17:57:54.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 -- 172.21.15.82:0/3664351552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc15f550 msgr2=0x7f83cc157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:54.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc15f550 0x7f83cc157740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f83c400efc0 tx=0x7f83c400c490 comp rx=0 tx=0).stop 2024-03-16T17:57:54.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 -- 172.21.15.82:0/3664351552 shutdown_connections 2024-03-16T17:57:54.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f83ac06b3e0 0x7f83ac06d8a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83cc15f550 0x7f83cc157740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83cc15ba20 0x7f83cc157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 --2- 172.21.15.82:0/3664351552 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83cc144400 0x7f83cc1408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:54.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 -- 172.21.15.82:0/3664351552 >> 172.21.15.82:0/3664351552 conn(0x7f83cc06ae70 msgr2=0x7f83cc0cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:54.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 -- 172.21.15.82:0/3664351552 shutdown_connections 2024-03-16T17:57:54.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:54.938+0000 7f83d4000640 1 -- 172.21.15.82:0/3664351552 wait complete. 2024-03-16T17:57:54.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:57:54.960 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:57:55.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.167+0000 7f43efe00640 1 Processor -- start 2024-03-16T17:57:55.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.167+0000 7f43efe00640 1 -- start start 2024-03-16T17:57:55.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.169+0000 7f43efe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43f006e010 0x7f43f0160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.169+0000 7f43efe00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f43f006e650 0x7f43f00cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.169+0000 7f43efe00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43f01458b0 con 0x7f43f01454d0 2024-03-16T17:57:55.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.169+0000 7f43efe00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f00cf3b0 con 0x7f43f006e010 2024-03-16T17:57:55.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.169+0000 7f43efe00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43f00cf690 con 0x7f43f006e650 2024-03-16T17:57:55.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.169+0000 7f43ee000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43f006e010 0x7f43f0160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.169+0000 7f43ee000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43f006e010 0x7f43f0160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55968/0 (socket says 172.21.15.82:55968) 2024-03-16T17:57:55.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.169+0000 7f43ee000640 1 -- 172.21.15.82:0/3903545724 learned_addr learned my addr 172.21.15.82:0/3903545724 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:55.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.169+0000 7f43ef400640 1 --2- 172.21.15.82:0/3903545724 >> v2:172.21.15.90:3300/0 conn(0x7f43f006e650 0x7f43f00cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.170+0000 7f43ee000640 1 -- 172.21.15.82:0/3903545724 >> v2:172.21.15.90:3300/0 conn(0x7f43f006e650 msgr2=0x7f43f00cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:55.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.170+0000 7f43ee000640 1 --2- 172.21.15.82:0/3903545724 >> v2:172.21.15.90:3300/0 conn(0x7f43f006e650 0x7f43f00cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.170+0000 7f43ee000640 1 -- 172.21.15.82:0/3903545724 >> v1:172.21.15.90:6789/0 conn(0x7f43f01454d0 legacy=0x7f43f006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:55.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.170+0000 7f43ee000640 1 -- 172.21.15.82:0/3903545724 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f015f7a0 con 0x7f43f006e010 2024-03-16T17:57:55.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.171+0000 7f43ee000640 1 --2- 172.21.15.82:0/3903545724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f006e010 0x7f43f0160280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43f006bbf0 tx=0x7f43dc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c81274e5473edad server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:55.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.171+0000 7f43ed600640 1 -- 172.21.15.82:0/3903545724 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc030860 con 0x7f43f006e010 2024-03-16T17:57:55.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.172+0000 7f43ed600640 1 -- 172.21.15.82:0/3903545724 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc0309c0 con 0x7f43f006e010 2024-03-16T17:57:55.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.172+0000 7f43ed600640 1 -- 172.21.15.82:0/3903545724 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc030d50 con 0x7f43f006e010 2024-03-16T17:57:55.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.172+0000 7f43efe00640 1 -- 172.21.15.82:0/3903545724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f006e010 msgr2=0x7f43f0160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:55.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.172+0000 7f43efe00640 1 --2- 172.21.15.82:0/3903545724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f006e010 0x7f43f0160280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43f006bbf0 tx=0x7f43dc02b9a0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.172+0000 7f43efe00640 1 -- 172.21.15.82:0/3903545724 shutdown_connections 2024-03-16T17:57:55.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.172+0000 7f43efe00640 1 --2- 172.21.15.82:0/3903545724 >> v2:172.21.15.90:3300/0 conn(0x7f43f006e650 0x7f43f00cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.172+0000 7f43efe00640 1 --2- 172.21.15.82:0/3903545724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f006e010 0x7f43f0160280 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.172+0000 7f43efe00640 1 -- 172.21.15.82:0/3903545724 >> 172.21.15.82:0/3903545724 conn(0x7f43f006ae70 msgr2=0x7f43f006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:55.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.172+0000 7f43efe00640 1 -- 172.21.15.82:0/3903545724 shutdown_connections 2024-03-16T17:57:55.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.172+0000 7f43efe00640 1 -- 172.21.15.82:0/3903545724 wait complete. 2024-03-16T17:57:55.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.174+0000 7f43efe00640 1 Processor -- start 2024-03-16T17:57:55.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.174+0000 7f43efe00640 1 -- start start 2024-03-16T17:57:55.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.175+0000 7f43efe00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f006e010 0x7f43f0135fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.175+0000 7f43efe00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43f006e650 0x7f43f00aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.175+0000 7f43efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f01454d0 0x7f43f00ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.175+0000 7f43efe00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f00ab7a0 con 0x7f43f01454d0 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.175+0000 7f43efe00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f0136500 con 0x7f43f006e010 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.175+0000 7f43efe00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f00aab20 con 0x7f43f006e650 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.175+0000 7f43ef400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f01454d0 0x7f43f00ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.175+0000 7f43ef400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f01454d0 0x7f43f00ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55982/0 (socket says 172.21.15.82:55982) 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.175+0000 7f43ef400640 1 -- 172.21.15.82:0/1909680493 learned_addr learned my addr 172.21.15.82:0/1909680493 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43eea00640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f006e010 0x7f43f0135fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43ef400640 1 -- 172.21.15.82:0/1909680493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43f006e650 msgr2=0x7f43f00aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43ee000640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43f006e650 0x7f43f00aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43ef400640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43f006e650 0x7f43f00aa060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43ef400640 1 -- 172.21.15.82:0/1909680493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f006e010 msgr2=0x7f43f0135fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43ef400640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f006e010 0x7f43f0135fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43ef400640 1 -- 172.21.15.82:0/1909680493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0094d0 con 0x7f43f01454d0 2024-03-16T17:57:55.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43ef400640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f01454d0 0x7f43f00ac6a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43e8007d90 tx=0x7f43e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:55.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43e7400640 1 -- 172.21.15.82:0/1909680493 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8014420 con 0x7f43f01454d0 2024-03-16T17:57:55.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43e7400640 1 -- 172.21.15.82:0/1909680493 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e80145a0 con 0x7f43f01454d0 2024-03-16T17:57:55.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43e7400640 1 -- 172.21.15.82:0/1909680493 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8014950 con 0x7f43f01454d0 2024-03-16T17:57:55.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43efe00640 1 -- 172.21.15.82:0/1909680493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f00aae00 con 0x7f43f01454d0 2024-03-16T17:57:55.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.176+0000 7f43efe00640 1 -- 172.21.15.82:0/1909680493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f00ab3c0 con 0x7f43f01454d0 2024-03-16T17:57:55.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.178+0000 7f43efe00640 1 -- 172.21.15.82:0/1909680493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f00ab8e0 con 0x7f43f01454d0 2024-03-16T17:57:55.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.178+0000 7f43e7400640 1 -- 172.21.15.82:0/1909680493 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f43e8014ab0 con 0x7f43f01454d0 2024-03-16T17:57:55.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.179+0000 7f43e7400640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f43d006b3b0 0x7f43d006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.179+0000 7f43e7400640 1 -- 172.21.15.82:0/1909680493 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43e808a560 con 0x7f43f01454d0 2024-03-16T17:57:55.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.180+0000 7f43eea00640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f43d006b3b0 0x7f43d006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.180+0000 7f43eea00640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f43d006b3b0 0x7f43d006d870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43e0007fd0 tx=0x7f43e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:55.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.184+0000 7f43e7400640 1 -- 172.21.15.82:0/1909680493 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f43f00ab8e0 con 0x7f43f01454d0 2024-03-16T17:57:55.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.382+0000 7f43efe00640 1 -- 172.21.15.82:0/1909680493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43f00ab8e0 con 0x7f43f01454d0 2024-03-16T17:57:55.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.382+0000 7f43e7400640 1 -- 172.21.15.82:0/1909680493 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43f00ab8e0 con 0x7f43f01454d0 2024-03-16T17:57:55.386 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:57:55.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 -- 172.21.15.82:0/1909680493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f43d006b3b0 msgr2=0x7f43d006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:55.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f43d006b3b0 0x7f43d006d870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43e0007fd0 tx=0x7f43e0009040 comp rx=0 tx=0).stop 2024-03-16T17:57:55.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 -- 172.21.15.82:0/1909680493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f01454d0 msgr2=0x7f43f00ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:55.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f01454d0 0x7f43f00ac6a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43e8007d90 tx=0x7f43e800e4b0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 -- 172.21.15.82:0/1909680493 shutdown_connections 2024-03-16T17:57:55.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f43d006b3b0 0x7f43d006d870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43f01454d0 0x7f43f00ac6a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f43f006e650 0x7f43f00aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 --2- 172.21.15.82:0/1909680493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43f006e010 0x7f43f0135fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 -- 172.21.15.82:0/1909680493 >> 172.21.15.82:0/1909680493 conn(0x7f43f006ae70 msgr2=0x7f43f006c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:55.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 -- 172.21.15.82:0/1909680493 shutdown_connections 2024-03-16T17:57:55.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.385+0000 7f43efe00640 1 -- 172.21.15.82:0/1909680493 wait complete. 2024-03-16T17:57:55.407 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:57:55.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.609+0000 7f35c3e00640 1 Processor -- start 2024-03-16T17:57:55.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.610+0000 7f35c3e00640 1 -- start start 2024-03-16T17:57:55.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35c415dd40 0x7f35c40d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f35c40b28d0 0x7f35c40b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 0x7f35c40d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c3e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c40d0e20 con 0x7f35c415dd40 2024-03-16T17:57:55.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c3e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c40d1790 con 0x7f35c40b28d0 2024-03-16T17:57:55.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c3e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c40d1a70 con 0x7f35c40d0490 2024-03-16T17:57:55.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c2a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35c415dd40 0x7f35c40d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c2a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35c415dd40 0x7f35c40d46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55996/0 (socket says 172.21.15.82:55996) 2024-03-16T17:57:55.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c2a00640 1 -- 172.21.15.82:0/3369068428 learned_addr learned my addr 172.21.15.82:0/3369068428 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:55.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c2000640 1 --2- 172.21.15.82:0/3369068428 >> v2:172.21.15.90:3300/0 conn(0x7f35c40b28d0 0x7f35c40b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c3400640 1 --2- 172.21.15.82:0/3369068428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 0x7f35c40d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c2a00640 1 -- 172.21.15.82:0/3369068428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 msgr2=0x7f35c40d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:55.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c2a00640 1 --2- 172.21.15.82:0/3369068428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 0x7f35c40d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c2a00640 1 -- 172.21.15.82:0/3369068428 >> v2:172.21.15.90:3300/0 conn(0x7f35c40b28d0 msgr2=0x7f35c40b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:55.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c2a00640 1 --2- 172.21.15.82:0/3369068428 >> v2:172.21.15.90:3300/0 conn(0x7f35c40b28d0 0x7f35c40b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c2a00640 1 -- 172.21.15.82:0/3369068428 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c40cf410 con 0x7f35c415dd40 2024-03-16T17:57:55.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.611+0000 7f35c3400640 1 --2- 172.21.15.82:0/3369068428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 0x7f35c40d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:55.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.612+0000 7f35c2a00640 1 --2- 172.21.15.82:0/3369068428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 0x7f35c40d46e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35c408d770 tx=0x7f35b402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7756174b8e4ebd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:55.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.613+0000 7f35c1600640 1 -- 172.21.15.82:0/3369068428 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40317d0 con 0x7f35c415dd40 2024-03-16T17:57:55.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.613+0000 7f35c1600640 1 -- 172.21.15.82:0/3369068428 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b4031930 con 0x7f35c415dd40 2024-03-16T17:57:55.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.613+0000 7f35c1600640 1 -- 172.21.15.82:0/3369068428 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4031cc0 con 0x7f35c415dd40 2024-03-16T17:57:55.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.613+0000 7f35c3e00640 1 -- 172.21.15.82:0/3369068428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 msgr2=0x7f35c40d46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:55.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.613+0000 7f35c3e00640 1 --2- 172.21.15.82:0/3369068428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 0x7f35c40d46e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35c408d770 tx=0x7f35b402b870 comp rx=0 tx=0).stop 2024-03-16T17:57:55.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.613+0000 7f35c3e00640 1 -- 172.21.15.82:0/3369068428 shutdown_connections 2024-03-16T17:57:55.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.613+0000 7f35c3e00640 1 --2- 172.21.15.82:0/3369068428 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 0x7f35c40d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.613+0000 7f35c3e00640 1 --2- 172.21.15.82:0/3369068428 >> v2:172.21.15.90:3300/0 conn(0x7f35c40b28d0 0x7f35c40b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.613+0000 7f35c3e00640 1 --2- 172.21.15.82:0/3369068428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 0x7f35c40d46e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.614+0000 7f35c3e00640 1 -- 172.21.15.82:0/3369068428 >> 172.21.15.82:0/3369068428 conn(0x7f35c40d3bc0 msgr2=0x7f35c40d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:55.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.614+0000 7f35c3e00640 1 -- 172.21.15.82:0/3369068428 shutdown_connections 2024-03-16T17:57:55.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.614+0000 7f35c3e00640 1 -- 172.21.15.82:0/3369068428 wait complete. 2024-03-16T17:57:55.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.615+0000 7f35c3e00640 1 Processor -- start 2024-03-16T17:57:55.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.615+0000 7f35c3e00640 1 -- start start 2024-03-16T17:57:55.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.616+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35c40b28d0 0x7f35c409d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.616+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 0x7f35c40b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.616+0000 7f35c3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 0x7f35c41539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.616+0000 7f35c3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4154520 con 0x7f35c415dd40 2024-03-16T17:57:55.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.616+0000 7f35c3e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c409fed0 con 0x7f35c40b28d0 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.616+0000 7f35c3e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c4153f20 con 0x7f35c40d0490 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 0x7f35c41539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 0x7f35c41539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56006/0 (socket says 172.21.15.82:56006) 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c3400640 1 -- 172.21.15.82:0/1555627630 learned_addr learned my addr 172.21.15.82:0/1555627630 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c2a00640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35c40b28d0 0x7f35c409d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c2000640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 0x7f35c40b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c3400640 1 -- 172.21.15.82:0/1555627630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 msgr2=0x7f35c40b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c3400640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 0x7f35c40b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c3400640 1 -- 172.21.15.82:0/1555627630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35c40b28d0 msgr2=0x7f35c409d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c3400640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35c40b28d0 0x7f35c409d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c3400640 1 -- 172.21.15.82:0/1555627630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b40094d0 con 0x7f35c415dd40 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c2000640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 0x7f35c40b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:55.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.617+0000 7f35c3400640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 0x7f35c41539e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00cef0 tx=0x7f35bc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:55.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.618+0000 7f35baa00640 1 -- 172.21.15.82:0/1555627630 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00d840 con 0x7f35c415dd40 2024-03-16T17:57:55.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.618+0000 7f35baa00640 1 -- 172.21.15.82:0/1555627630 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc00d9c0 con 0x7f35c415dd40 2024-03-16T17:57:55.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.618+0000 7f35baa00640 1 -- 172.21.15.82:0/1555627630 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00dd70 con 0x7f35c415dd40 2024-03-16T17:57:55.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.618+0000 7f35c3e00640 1 -- 172.21.15.82:0/1555627630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c415e2a0 con 0x7f35c415dd40 2024-03-16T17:57:55.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.618+0000 7f35c3e00640 1 -- 172.21.15.82:0/1555627630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c409f880 con 0x7f35c415dd40 2024-03-16T17:57:55.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.619+0000 7f35c3e00640 1 -- 172.21.15.82:0/1555627630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c40a0080 con 0x7f35c415dd40 2024-03-16T17:57:55.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.620+0000 7f35baa00640 1 -- 172.21.15.82:0/1555627630 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f35bc01e3b0 con 0x7f35c415dd40 2024-03-16T17:57:55.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.621+0000 7f35baa00640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f35a006b3e0 0x7f35a006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:55.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.621+0000 7f35baa00640 1 -- 172.21.15.82:0/1555627630 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35bc08a700 con 0x7f35c415dd40 2024-03-16T17:57:55.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.621+0000 7f35c2a00640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f35a006b3e0 0x7f35a006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:55.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.622+0000 7f35c2a00640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f35a006b3e0 0x7f35a006d8a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f35b4036000 tx=0x7f35b403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:55.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.626+0000 7f35baa00640 1 -- 172.21.15.82:0/1555627630 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f35c40a0080 con 0x7f35c415dd40 2024-03-16T17:57:55.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.819+0000 7f35c3e00640 1 -- 172.21.15.82:0/1555627630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f35c40a0040 con 0x7f35c415dd40 2024-03-16T17:57:55.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.820+0000 7f35baa00640 1 -- 172.21.15.82:0/1555627630 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f35bc055380 con 0x7f35c415dd40 2024-03-16T17:57:55.824 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:57:55.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.822+0000 7f35c3e00640 1 -- 172.21.15.82:0/1555627630 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f35a006b3e0 msgr2=0x7f35a006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:55.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.822+0000 7f35c3e00640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f35a006b3e0 0x7f35a006d8a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f35b4036000 tx=0x7f35b403d040 comp rx=0 tx=0).stop 2024-03-16T17:57:55.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.823+0000 7f35c3e00640 1 -- 172.21.15.82:0/1555627630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 msgr2=0x7f35c41539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:55.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.823+0000 7f35c3e00640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 0x7f35c41539e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00cef0 tx=0x7f35bc00a710 comp rx=0 tx=0).stop 2024-03-16T17:57:55.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.823+0000 7f35c3e00640 1 -- 172.21.15.82:0/1555627630 shutdown_connections 2024-03-16T17:57:55.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.823+0000 7f35c3e00640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f35a006b3e0 0x7f35a006d8a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.823+0000 7f35c3e00640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35c415dd40 0x7f35c41539e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.823+0000 7f35c3e00640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f35c40d0490 0x7f35c40b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.823+0000 7f35c3e00640 1 --2- 172.21.15.82:0/1555627630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35c40b28d0 0x7f35c409d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:55.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.823+0000 7f35c3e00640 1 -- 172.21.15.82:0/1555627630 >> 172.21.15.82:0/1555627630 conn(0x7f35c40d3bc0 msgr2=0x7f35c40ce720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:55.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.823+0000 7f35c3e00640 1 -- 172.21.15.82:0/1555627630 shutdown_connections 2024-03-16T17:57:55.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:55.823+0000 7f35c3e00640 1 -- 172.21.15.82:0/1555627630 wait complete. 2024-03-16T17:57:55.844 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:57:56.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.032+0000 7f9942e00640 1 Processor -- start 2024-03-16T17:57:56.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.032+0000 7f9942e00640 1 -- start start 2024-03-16T17:57:56.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.034+0000 7f9942e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f993c144330 0x7f993c070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.034+0000 7f9942e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f993c0cc380 0x7f993c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.034+0000 7f9942e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993c0ce8f0 con 0x7f993c0714f0 2024-03-16T17:57:56.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.034+0000 7f9942e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c0ccd50 con 0x7f993c0cc380 2024-03-16T17:57:56.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.034+0000 7f9942e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c15f2d0 con 0x7f993c144330 2024-03-16T17:57:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.034+0000 7f9940e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f993c0cc380 0x7f993c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.034+0000 7f9940e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f993c0cc380 0x7f993c0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56014/0 (socket says 172.21.15.82:56014) 2024-03-16T17:57:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.034+0000 7f9940e00640 1 -- 172.21.15.82:0/4076665523 learned_addr learned my addr 172.21.15.82:0/4076665523 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.034+0000 7f993be00640 1 --2- 172.21.15.82:0/4076665523 >> v2:172.21.15.90:3300/0 conn(0x7f993c144330 0x7f993c070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.035+0000 7f9940e00640 1 -- 172.21.15.82:0/4076665523 >> v2:172.21.15.90:3300/0 conn(0x7f993c144330 msgr2=0x7f993c070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.035+0000 7f9940e00640 1 --2- 172.21.15.82:0/4076665523 >> v2:172.21.15.90:3300/0 conn(0x7f993c144330 0x7f993c070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.035+0000 7f9940e00640 1 -- 172.21.15.82:0/4076665523 >> v1:172.21.15.90:6789/0 conn(0x7f993c0714f0 legacy=0x7f993c15e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.035+0000 7f9940e00640 1 -- 172.21.15.82:0/4076665523 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c15f5b0 con 0x7f993c0cc380 2024-03-16T17:57:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.036+0000 7f9940e00640 1 --2- 172.21.15.82:0/4076665523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 0x7f993c0cc810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f993c0738c0 tx=0x7f993402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e631129ba6b8b1b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.036+0000 7f993aa00640 1 -- 172.21.15.82:0/4076665523 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993402ca80 con 0x7f993c0cc380 2024-03-16T17:57:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.036+0000 7f993aa00640 1 -- 172.21.15.82:0/4076665523 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993402cbe0 con 0x7f993c0cc380 2024-03-16T17:57:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.036+0000 7f993aa00640 1 -- 172.21.15.82:0/4076665523 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340343f0 con 0x7f993c0cc380 2024-03-16T17:57:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.036+0000 7f9942e00640 1 -- 172.21.15.82:0/4076665523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 msgr2=0x7f993c0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.036+0000 7f9942e00640 1 --2- 172.21.15.82:0/4076665523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 0x7f993c0cc810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f993c0738c0 tx=0x7f993402bee0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.037+0000 7f9942e00640 1 -- 172.21.15.82:0/4076665523 shutdown_connections 2024-03-16T17:57:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.037+0000 7f9942e00640 1 --2- 172.21.15.82:0/4076665523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 0x7f993c0cc810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.037+0000 7f9942e00640 1 --2- 172.21.15.82:0/4076665523 >> v2:172.21.15.90:3300/0 conn(0x7f993c144330 0x7f993c070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.037+0000 7f9942e00640 1 -- 172.21.15.82:0/4076665523 >> 172.21.15.82:0/4076665523 conn(0x7f993c0b1640 msgr2=0x7f993c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.037+0000 7f9942e00640 1 -- 172.21.15.82:0/4076665523 shutdown_connections 2024-03-16T17:57:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.037+0000 7f9942e00640 1 -- 172.21.15.82:0/4076665523 wait complete. 2024-03-16T17:57:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.038+0000 7f9942e00640 1 Processor -- start 2024-03-16T17:57:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.038+0000 7f9942e00640 1 -- start start 2024-03-16T17:57:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.039+0000 7f9942e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c0714f0 0x7f993c0b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.039+0000 7f9942e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 0x7f993c158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.039+0000 7f9942e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f993c144330 0x7f993c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.039+0000 7f9942e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c1f4ea0 con 0x7f993c0cc380 2024-03-16T17:57:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.039+0000 7f9942e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c1f5e40 con 0x7f993c0714f0 2024-03-16T17:57:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.039+0000 7f9942e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c0b3f40 con 0x7f993c144330 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.039+0000 7f993b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 0x7f993c158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.039+0000 7f993b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 0x7f993c158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56020/0 (socket says 172.21.15.82:56020) 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.039+0000 7f993b400640 1 -- 172.21.15.82:0/1463953337 learned_addr learned my addr 172.21.15.82:0/1463953337 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f993b400640 1 -- 172.21.15.82:0/1463953337 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f993c144330 msgr2=0x7f993c0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f993b400640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f993c144330 0x7f993c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f993be00640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c0714f0 0x7f993c0b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f993b400640 1 -- 172.21.15.82:0/1463953337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c0714f0 msgr2=0x7f993c0b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f993b400640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c0714f0 0x7f993c0b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f993b400640 1 -- 172.21.15.82:0/1463953337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9934009b30 con 0x7f993c0cc380 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f993b400640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 0x7f993c158710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f992800b3f0 tx=0x7f992800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f9933e00640 1 -- 172.21.15.82:0/1463953337 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928011830 con 0x7f993c0cc380 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f9933e00640 1 -- 172.21.15.82:0/1463953337 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9928011990 con 0x7f993c0cc380 2024-03-16T17:57:56.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f9933e00640 1 -- 172.21.15.82:0/1463953337 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928011d40 con 0x7f993c0cc380 2024-03-16T17:57:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.040+0000 7f9942e00640 1 -- 172.21.15.82:0/1463953337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c1f5080 con 0x7f993c0cc380 2024-03-16T17:57:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.041+0000 7f9942e00640 1 -- 172.21.15.82:0/1463953337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c1f5640 con 0x7f993c0cc380 2024-03-16T17:57:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.041+0000 7f9942e00640 1 -- 172.21.15.82:0/1463953337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c067f30 con 0x7f993c0cc380 2024-03-16T17:57:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.042+0000 7f9933e00640 1 -- 172.21.15.82:0/1463953337 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f992801a400 con 0x7f993c0cc380 2024-03-16T17:57:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.043+0000 7f9933e00640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f991c06b3b0 0x7f991c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.043+0000 7f9933e00640 1 -- 172.21.15.82:0/1463953337 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9928089d00 con 0x7f993c0cc380 2024-03-16T17:57:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.044+0000 7f993be00640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f991c06b3b0 0x7f991c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.044+0000 7f993be00640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f991c06b3b0 0x7f991c06d870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f992c006fd0 tx=0x7f992c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:56.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.048+0000 7f9933e00640 1 -- 172.21.15.82:0/1463953337 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9928054940 con 0x7f993c0cc380 2024-03-16T17:57:56.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.244+0000 7f9942e00640 1 -- 172.21.15.82:0/1463953337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f993c072bc0 con 0x7f993c0cc380 2024-03-16T17:57:56.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.245+0000 7f9933e00640 1 -- 172.21.15.82:0/1463953337 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99280150a0 con 0x7f993c0cc380 2024-03-16T17:57:56.249 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 -- 172.21.15.82:0/1463953337 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f991c06b3b0 msgr2=0x7f991c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f991c06b3b0 0x7f991c06d870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f992c006fd0 tx=0x7f992c008040 comp rx=0 tx=0).stop 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 -- 172.21.15.82:0/1463953337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 msgr2=0x7f993c158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 0x7f993c158710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f992800b3f0 tx=0x7f992800b8b0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 -- 172.21.15.82:0/1463953337 shutdown_connections 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f991c06b3b0 0x7f991c06d870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f993c144330 0x7f993c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f993c0cc380 0x7f993c158710 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 --2- 172.21.15.82:0/1463953337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f993c0714f0 0x7f993c0b2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 -- 172.21.15.82:0/1463953337 >> 172.21.15.82:0/1463953337 conn(0x7f993c0b1640 msgr2=0x7f993c15e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 -- 172.21.15.82:0/1463953337 shutdown_connections 2024-03-16T17:57:56.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.248+0000 7f9942e00640 1 -- 172.21.15.82:0/1463953337 wait complete. 2024-03-16T17:57:56.270 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:57:56.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.476+0000 7f5ca8000640 1 Processor -- start 2024-03-16T17:57:56.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.476+0000 7f5ca8000640 1 -- start start 2024-03-16T17:57:56.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5ca8000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ca00d00a0 0x7f5ca015b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5ca8000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca015e820 con 0x7f5ca015f550 2024-03-16T17:57:56.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5ca8000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca015c080 con 0x7f5ca0144400 2024-03-16T17:57:56.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5ca8000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca015f130 con 0x7f5ca00d00a0 2024-03-16T17:57:56.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5ca4c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ca00d00a0 0x7f5ca015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5ca6000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5ca015f550 0x7f5ca015b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49104/0 (socket says 172.21.15.82:49104) 2024-03-16T17:57:56.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5ca6000640 1 -- 172.21.15.82:0/2091973831 learned_addr learned my addr 172.21.15.82:0/2091973831 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:56.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5ca4c00640 1 -- 172.21.15.82:0/2091973831 >> v1:172.21.15.90:6789/0 conn(0x7f5ca0144400 legacy=0x7f5ca00cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:56.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5ca4c00640 1 -- 172.21.15.82:0/2091973831 >> v1:172.21.15.82:6789/0 conn(0x7f5ca015f550 legacy=0x7f5ca015b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5ca4c00640 1 -- 172.21.15.82:0/2091973831 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca015ea70 con 0x7f5ca00d00a0 2024-03-16T17:57:56.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.478+0000 7f5c9fe00640 1 -- 172.21.15.82:0/2091973831 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597769118 0 0) 0x7f5ca015e820 con 0x7f5ca015f550 2024-03-16T17:57:56.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.480+0000 7f5ca4c00640 1 --2- 172.21.15.82:0/2091973831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca00d00a0 0x7f5ca015b310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0097a0 tx=0x7f5c8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de51eaaaaec6caf7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:56.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.480+0000 7f5c9fe00640 1 -- 172.21.15.82:0/2091973831 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c02e070 con 0x7f5ca00d00a0 2024-03-16T17:57:56.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.480+0000 7f5c9fe00640 1 -- 172.21.15.82:0/2091973831 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c0315a0 con 0x7f5ca00d00a0 2024-03-16T17:57:56.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.480+0000 7f5c9fe00640 1 -- 172.21.15.82:0/2091973831 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c031930 con 0x7f5ca00d00a0 2024-03-16T17:57:56.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.480+0000 7f5ca8000640 1 -- 172.21.15.82:0/2091973831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca00d00a0 msgr2=0x7f5ca015b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.480+0000 7f5ca8000640 1 --2- 172.21.15.82:0/2091973831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca00d00a0 0x7f5ca015b310 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c0097a0 tx=0x7f5c8c02b840 comp rx=0 tx=0).stop 2024-03-16T17:57:56.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.481+0000 7f5ca8000640 1 -- 172.21.15.82:0/2091973831 shutdown_connections 2024-03-16T17:57:56.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.481+0000 7f5ca8000640 1 --2- 172.21.15.82:0/2091973831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca00d00a0 0x7f5ca015b310 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.481+0000 7f5ca8000640 1 -- 172.21.15.82:0/2091973831 >> 172.21.15.82:0/2091973831 conn(0x7f5ca006ae70 msgr2=0x7f5ca006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:56.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.481+0000 7f5ca8000640 1 -- 172.21.15.82:0/2091973831 shutdown_connections 2024-03-16T17:57:56.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.481+0000 7f5ca8000640 1 -- 172.21.15.82:0/2091973831 wait complete. 2024-03-16T17:57:56.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.482+0000 7f5ca8000640 1 Processor -- start 2024-03-16T17:57:56.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.482+0000 7f5ca8000640 1 -- start start 2024-03-16T17:57:56.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.483+0000 7f5ca8000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ca00d00a0 0x7f5ca00af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.483+0000 7f5ca8000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca0144400 0x7f5ca013d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.483+0000 7f5ca8000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ca015f550 0x7f5ca013f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.483+0000 7f5ca8000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca01429e0 con 0x7f5ca0144400 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.483+0000 7f5ca8000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca0142c50 con 0x7f5ca00d00a0 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.483+0000 7f5ca8000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca0142f30 con 0x7f5ca015f550 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.484+0000 7f5ca4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca0144400 0x7f5ca013d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.484+0000 7f5ca4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca0144400 0x7f5ca013d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56028/0 (socket says 172.21.15.82:56028) 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.484+0000 7f5ca4c00640 1 -- 172.21.15.82:0/2282133154 learned_addr learned my addr 172.21.15.82:0/2282133154 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.484+0000 7f5ca5600640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ca00d00a0 0x7f5ca00af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.484+0000 7f5ca4c00640 1 -- 172.21.15.82:0/2282133154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ca015f550 msgr2=0x7f5ca013f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.484+0000 7f5ca4c00640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ca015f550 0x7f5ca013f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.484+0000 7f5ca4c00640 1 -- 172.21.15.82:0/2282133154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ca00d00a0 msgr2=0x7f5ca00af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.484+0000 7f5ca4c00640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ca00d00a0 0x7f5ca00af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.484+0000 7f5ca4c00640 1 -- 172.21.15.82:0/2282133154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c009400 con 0x7f5ca0144400 2024-03-16T17:57:56.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.484+0000 7f5ca4c00640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca0144400 0x7f5ca013d140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c034040 tx=0x7f5c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:56.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.485+0000 7f5c9d600640 1 -- 172.21.15.82:0/2282133154 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c032900 con 0x7f5ca0144400 2024-03-16T17:57:56.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.485+0000 7f5c9d600640 1 -- 172.21.15.82:0/2282133154 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c032a60 con 0x7f5ca0144400 2024-03-16T17:57:56.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.485+0000 7f5c9d600640 1 -- 172.21.15.82:0/2282133154 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c032e10 con 0x7f5ca0144400 2024-03-16T17:57:56.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.485+0000 7f5ca8000640 1 -- 172.21.15.82:0/2282133154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca0143210 con 0x7f5ca0144400 2024-03-16T17:57:56.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.485+0000 7f5ca8000640 1 -- 172.21.15.82:0/2282133154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca013d880 con 0x7f5ca0144400 2024-03-16T17:57:56.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.486+0000 7f5ca8000640 1 -- 172.21.15.82:0/2282133154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca013da70 con 0x7f5ca0144400 2024-03-16T17:57:56.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.487+0000 7f5c9d600640 1 -- 172.21.15.82:0/2282133154 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f5c8c03b3f0 con 0x7f5ca0144400 2024-03-16T17:57:56.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.487+0000 7f5c9d600640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5c8006b3e0 0x7f5c8006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.488+0000 7f5c9d600640 1 -- 172.21.15.82:0/2282133154 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c8c0abca0 con 0x7f5ca0144400 2024-03-16T17:57:56.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.488+0000 7f5ca5600640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5c8006b3e0 0x7f5c8006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.488+0000 7f5ca5600640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5c8006b3e0 0x7f5c8006d8a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5c94006fd0 tx=0x7f5c94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:56.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.492+0000 7f5c9d600640 1 -- 172.21.15.82:0/2282133154 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5ca013da70 con 0x7f5ca0144400 2024-03-16T17:57:56.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.715+0000 7f5ca8000640 1 -- 172.21.15.82:0/2282133154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ca0143730 con 0x7f5ca0144400 2024-03-16T17:57:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.716+0000 7f5c9d600640 1 -- 172.21.15.82:0/2282133154 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4968 (secure 0 0 0) 0x7f5ca0143730 con 0x7f5ca0144400 2024-03-16T17:57:56.720 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:56.720 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2570794110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2570794110},{"type":"v1","addr":"172.21.15.90:6835","nonce":2570794110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3135739259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3135739259},{"type":"v1","addr":"172.21.15.82:6837","nonce":3135739259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3721205060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3721205060},{"type":"v1","addr":"172.21.15.82:6835","nonce":3721205060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:57:40.797669+0000","modified":"2024-03-16T17:57:43.865338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/732041173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":732041173},{"type":"v1","addr":"172.21.15.90:6833","nonce":732041173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4267,"qdb_cluster":[4267]},"id":1}]} 2024-03-16T17:57:56.720 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-03-16T17:57:56.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 -- 172.21.15.82:0/2282133154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5c8006b3e0 msgr2=0x7f5c8006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5c8006b3e0 0x7f5c8006d8a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5c94006fd0 tx=0x7f5c94008040 comp rx=0 tx=0).stop 2024-03-16T17:57:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 -- 172.21.15.82:0/2282133154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca0144400 msgr2=0x7f5ca013d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca0144400 0x7f5ca013d140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c034040 tx=0x7f5c8c0046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 -- 172.21.15.82:0/2282133154 shutdown_connections 2024-03-16T17:57:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5c8006b3e0 0x7f5c8006d8a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ca015f550 0x7f5ca013f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ca0144400 0x7f5ca013d140 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 --2- 172.21.15.82:0/2282133154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ca00d00a0 0x7f5ca00af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 -- 172.21.15.82:0/2282133154 >> 172.21.15.82:0/2282133154 conn(0x7f5ca006ae70 msgr2=0x7f5ca006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:56.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 -- 172.21.15.82:0/2282133154 shutdown_connections 2024-03-16T17:57:56.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.719+0000 7f5ca8000640 1 -- 172.21.15.82:0/2282133154 wait complete. 2024-03-16T17:57:56.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:57:56.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.947+0000 7f59c7000640 1 Processor -- start 2024-03-16T17:57:56.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.947+0000 7f59c7000640 1 -- start start 2024-03-16T17:57:56.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59c7000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f59c015df00 0x7f59c00d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59c7000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59c00b32c0 0x7f59c00d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59c7000640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f59c015e2e0 0x7f59c00d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59c7000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c00cfe40 con 0x7f59c015df00 2024-03-16T17:57:56.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59c7000640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c00d4940 con 0x7f59c015e2e0 2024-03-16T17:57:56.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59c7000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00b3780 con 0x7f59c00b32c0 2024-03-16T17:57:56.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59bfe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f59c015df00 0x7f59c00d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59bfe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f59c015df00 0x7f59c00d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56042/0 (socket says 172.21.15.82:56042) 2024-03-16T17:57:56.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59bfe00640 1 -- 172.21.15.82:0/3558013256 learned_addr learned my addr 172.21.15.82:0/3558013256 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:56.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59bf400640 1 --2- 172.21.15.82:0/3558013256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59c00b32c0 0x7f59c00d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.949+0000 7f59c5000640 1 --2- 172.21.15.82:0/3558013256 >> v2:172.21.15.90:3300/0 conn(0x7f59c015e2e0 0x7f59c00d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.950+0000 7f59bfe00640 1 -- 172.21.15.82:0/3558013256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59c00b32c0 msgr2=0x7f59c00d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:56.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.950+0000 7f59bfe00640 1 --2- 172.21.15.82:0/3558013256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59c00b32c0 0x7f59c00d2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.950+0000 7f59bfe00640 1 -- 172.21.15.82:0/3558013256 >> v2:172.21.15.90:3300/0 conn(0x7f59c015e2e0 msgr2=0x7f59c00d28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:56.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.950+0000 7f59bfe00640 1 --2- 172.21.15.82:0/3558013256 >> v2:172.21.15.90:3300/0 conn(0x7f59c015e2e0 0x7f59c00d28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.950+0000 7f59bfe00640 1 -- 172.21.15.82:0/3558013256 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00b3a60 con 0x7f59c015df00 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.951+0000 7f59bfe00640 1 --2- 172.21.15.82:0/3558013256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015df00 0x7f59c00d0dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f59c00ce460 tx=0x7f59b002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c33af7119b829e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.951+0000 7f59bea00640 1 -- 172.21.15.82:0/3558013256 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b002cb00 con 0x7f59c015df00 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.951+0000 7f59bea00640 1 -- 172.21.15.82:0/3558013256 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b002cc60 con 0x7f59c015df00 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.951+0000 7f59bea00640 1 -- 172.21.15.82:0/3558013256 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00343f0 con 0x7f59c015df00 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.951+0000 7f59c7000640 1 -- 172.21.15.82:0/3558013256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015df00 msgr2=0x7f59c00d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.951+0000 7f59c7000640 1 --2- 172.21.15.82:0/3558013256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015df00 0x7f59c00d0dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f59c00ce460 tx=0x7f59b002bf60 comp rx=0 tx=0).stop 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.952+0000 7f59c7000640 1 -- 172.21.15.82:0/3558013256 shutdown_connections 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.952+0000 7f59c7000640 1 --2- 172.21.15.82:0/3558013256 >> v2:172.21.15.90:3300/0 conn(0x7f59c015e2e0 0x7f59c00d28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.952+0000 7f59c7000640 1 --2- 172.21.15.82:0/3558013256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59c00b32c0 0x7f59c00d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.952+0000 7f59c7000640 1 --2- 172.21.15.82:0/3558013256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015df00 0x7f59c00d0dc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.952+0000 7f59c7000640 1 -- 172.21.15.82:0/3558013256 >> 172.21.15.82:0/3558013256 conn(0x7f59c00d3d80 msgr2=0x7f59c00d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.952+0000 7f59c7000640 1 -- 172.21.15.82:0/3558013256 shutdown_connections 2024-03-16T17:57:56.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.952+0000 7f59c7000640 1 -- 172.21.15.82:0/3558013256 wait complete. 2024-03-16T17:57:56.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.954+0000 7f59c7000640 1 Processor -- start 2024-03-16T17:57:56.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.954+0000 7f59c7000640 1 -- start start 2024-03-16T17:57:56.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59c7000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59c00b32c0 0x7f59c00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59c7000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59c015df00 0x7f59c01f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59c7000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015e2e0 0x7f59c015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59c7000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c01f4720 con 0x7f59c015e2e0 2024-03-16T17:57:56.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59c7000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c01f4ca0 con 0x7f59c015df00 2024-03-16T17:57:56.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59c7000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00b4b80 con 0x7f59c00b32c0 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59c5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015e2e0 0x7f59c015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59c5000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015e2e0 0x7f59c015b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56052/0 (socket says 172.21.15.82:56052) 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59c5000640 1 -- 172.21.15.82:0/1323399476 learned_addr learned my addr 172.21.15.82:0/1323399476 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59bf400640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59c015df00 0x7f59c01f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.955+0000 7f59bfe00640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59c00b32c0 0x7f59c00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59c5000640 1 -- 172.21.15.82:0/1323399476 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59c00b32c0 msgr2=0x7f59c00c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59c5000640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59c00b32c0 0x7f59c00c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59c5000640 1 -- 172.21.15.82:0/1323399476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59c015df00 msgr2=0x7f59c01f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59c5000640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59c015df00 0x7f59c01f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59c5000640 1 -- 172.21.15.82:0/1323399476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b0009b30 con 0x7f59c015e2e0 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59c5000640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015e2e0 0x7f59c015b320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f59b800d5d0 tx=0x7f59b800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59b7e00640 1 -- 172.21.15.82:0/1323399476 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b80125f0 con 0x7f59c015e2e0 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59b7e00640 1 -- 172.21.15.82:0/1323399476 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b8012750 con 0x7f59c015e2e0 2024-03-16T17:57:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59b7e00640 1 -- 172.21.15.82:0/1323399476 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8012ae0 con 0x7f59c015e2e0 2024-03-16T17:57:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59c7000640 1 -- 172.21.15.82:0/1323399476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c00b4e60 con 0x7f59c015e2e0 2024-03-16T17:57:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.956+0000 7f59c7000640 1 -- 172.21.15.82:0/1323399476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c01f4930 con 0x7f59c015e2e0 2024-03-16T17:57:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.957+0000 7f59c7000640 1 -- 172.21.15.82:0/1323399476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c01f4e30 con 0x7f59c015e2e0 2024-03-16T17:57:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.958+0000 7f59b7e00640 1 -- 172.21.15.82:0/1323399476 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f59b8012c40 con 0x7f59c015e2e0 2024-03-16T17:57:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.959+0000 7f59b7e00640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f599c06b3b0 0x7f599c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.959+0000 7f59b7e00640 1 -- 172.21.15.82:0/1323399476 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59b8089ff0 con 0x7f59c015e2e0 2024-03-16T17:57:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.959+0000 7f59bfe00640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f599c06b3b0 0x7f599c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.960+0000 7f59bfe00640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f599c06b3b0 0x7f599c06d870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59b0035000 tx=0x7f59b0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:56.963+0000 7f59b7e00640 1 -- 172.21.15.82:0/1323399476 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f59b80bde10 con 0x7f59c015e2e0 2024-03-16T17:57:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.145+0000 7f59c7000640 1 -- 172.21.15.82:0/1323399476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59c0067f30 con 0x7f59c015e2e0 2024-03-16T17:57:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.148+0000 7f59b7e00640 1 -- 172.21.15.82:0/1323399476 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f59b8054c00 con 0x7f59c015e2e0 2024-03-16T17:57:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:57.152 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:57:57.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 -- 172.21.15.82:0/1323399476 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f599c06b3b0 msgr2=0x7f599c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f599c06b3b0 0x7f599c06d870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59b0035000 tx=0x7f59b0030000 comp rx=0 tx=0).stop 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 -- 172.21.15.82:0/1323399476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015e2e0 msgr2=0x7f59c015b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015e2e0 0x7f59c015b320 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f59b800d5d0 tx=0x7f59b800da90 comp rx=0 tx=0).stop 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 -- 172.21.15.82:0/1323399476 shutdown_connections 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f599c06b3b0 0x7f599c06d870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59c015e2e0 0x7f59c015b320 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59c015df00 0x7f59c01f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 --2- 172.21.15.82:0/1323399476 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59c00b32c0 0x7f59c00c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 -- 172.21.15.82:0/1323399476 >> 172.21.15.82:0/1323399476 conn(0x7f59c00d3d80 msgr2=0x7f59c00b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 -- 172.21.15.82:0/1323399476 shutdown_connections 2024-03-16T17:57:57.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.151+0000 7f59c7000640 1 -- 172.21.15.82:0/1323399476 wait complete. 2024-03-16T17:57:57.171 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:57:57.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.372+0000 7f6673200640 1 Processor -- start 2024-03-16T17:57:57.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.373+0000 7f6673200640 1 -- start start 2024-03-16T17:57:57.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.374+0000 7f6673200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f666c15e180 0x7f666c0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.374+0000 7f6673200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f666c0d0e70 0x7f666c0d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.374+0000 7f6673200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c15e560 con 0x7f666c15f110 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.374+0000 7f6673200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666c0cf0d0 con 0x7f666c0d0e70 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.374+0000 7f6673200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666c0cf3b0 con 0x7f666c15e180 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.375+0000 7f6671200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f666c0d0e70 0x7f666c0d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.375+0000 7f6671200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f666c0d0e70 0x7f666c0d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56066/0 (socket says 172.21.15.82:56066) 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.375+0000 7f6671200640 1 -- 172.21.15.82:0/2714396586 learned_addr learned my addr 172.21.15.82:0/2714396586 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.375+0000 7f666b400640 1 --1- 172.21.15.82:0/2714396586 >> v1:172.21.15.82:6789/0 conn(0x7f666c15f110 0x7f666c0d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49116/0 (socket says 172.21.15.82:49116) 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.375+0000 7f666be00640 1 --2- 172.21.15.82:0/2714396586 >> v2:172.21.15.90:3300/0 conn(0x7f666c15e180 0x7f666c0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.375+0000 7f6671200640 1 -- 172.21.15.82:0/2714396586 >> v2:172.21.15.90:3300/0 conn(0x7f666c15e180 msgr2=0x7f666c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.375+0000 7f6671200640 1 --2- 172.21.15.82:0/2714396586 >> v2:172.21.15.90:3300/0 conn(0x7f666c15e180 0x7f666c0cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.375+0000 7f6671200640 1 -- 172.21.15.82:0/2714396586 >> v1:172.21.15.82:6789/0 conn(0x7f666c15f110 legacy=0x7f666c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.375+0000 7f6671200640 1 -- 172.21.15.82:0/2714396586 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c0cf690 con 0x7f666c0d0e70 2024-03-16T17:57:57.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.375+0000 7f666aa00640 1 -- 172.21.15.82:0/2714396586 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765029965 0 0) 0x7f6658001110 con 0x7f666c15f110 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.376+0000 7f6671200640 1 --2- 172.21.15.82:0/2714396586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 0x7f666c0d1bd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f666c0b6a50 tx=0x7f666402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ee6a130d33c035 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.376+0000 7f666aa00640 1 -- 172.21.15.82:0/2714396586 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666402cb20 con 0x7f666c0d0e70 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.376+0000 7f666aa00640 1 -- 172.21.15.82:0/2714396586 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666402cc80 con 0x7f666c0d0e70 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.376+0000 7f666aa00640 1 -- 172.21.15.82:0/2714396586 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66640343f0 con 0x7f666c0d0e70 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.377+0000 7f6673200640 1 -- 172.21.15.82:0/2714396586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 msgr2=0x7f666c0d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.377+0000 7f6673200640 1 --2- 172.21.15.82:0/2714396586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 0x7f666c0d1bd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f666c0b6a50 tx=0x7f666402bf80 comp rx=0 tx=0).stop 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.377+0000 7f6673200640 1 -- 172.21.15.82:0/2714396586 shutdown_connections 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.377+0000 7f6673200640 1 --2- 172.21.15.82:0/2714396586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 0x7f666c0d1bd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.377+0000 7f6673200640 1 --2- 172.21.15.82:0/2714396586 >> v2:172.21.15.90:3300/0 conn(0x7f666c15e180 0x7f666c0cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.377+0000 7f6673200640 1 -- 172.21.15.82:0/2714396586 >> 172.21.15.82:0/2714396586 conn(0x7f666c0d3e90 msgr2=0x7f666c0d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.377+0000 7f6673200640 1 -- 172.21.15.82:0/2714396586 shutdown_connections 2024-03-16T17:57:57.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.377+0000 7f6673200640 1 -- 172.21.15.82:0/2714396586 wait complete. 2024-03-16T17:57:57.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.379+0000 7f6673200640 1 Processor -- start 2024-03-16T17:57:57.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.379+0000 7f6673200640 1 -- start start 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.380+0000 7f6673200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 0x7f666c1579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.380+0000 7f6673200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f666c15e180 0x7f666c154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.380+0000 7f6673200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f666c15f110 0x7f666c06c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.380+0000 7f6673200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c0cc3d0 con 0x7f666c0d0e70 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.380+0000 7f6673200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c155520 con 0x7f666c15f110 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.380+0000 7f6673200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666c1580c0 con 0x7f666c15e180 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.380+0000 7f666be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 0x7f666c1579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.380+0000 7f666be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 0x7f666c1579e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56074/0 (socket says 172.21.15.82:56074) 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.380+0000 7f666be00640 1 -- 172.21.15.82:0/2788956634 learned_addr learned my addr 172.21.15.82:0/2788956634 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f6671200640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f666c15f110 0x7f666c06c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f666be00640 1 -- 172.21.15.82:0/2788956634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f666c15e180 msgr2=0x7f666c154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:57.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f666b400640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f666c15e180 0x7f666c154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f666be00640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f666c15e180 0x7f666c154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f666be00640 1 -- 172.21.15.82:0/2788956634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f666c15f110 msgr2=0x7f666c06c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f666be00640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f666c15f110 0x7f666c06c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f666be00640 1 -- 172.21.15.82:0/2788956634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664009b30 con 0x7f666c0d0e70 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f6671200640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f666c15f110 0x7f666c06c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f666be00640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 0x7f666c1579e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f665c00d830 tx=0x7f665c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f6663e00640 1 -- 172.21.15.82:0/2788956634 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00b6c0 con 0x7f666c0d0e70 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f6663e00640 1 -- 172.21.15.82:0/2788956634 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c00b820 con 0x7f666c0d0e70 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f6663e00640 1 -- 172.21.15.82:0/2788956634 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c00bbb0 con 0x7f666c0d0e70 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.381+0000 7f6673200640 1 -- 172.21.15.82:0/2788956634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666c0cc620 con 0x7f666c0d0e70 2024-03-16T17:57:57.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.382+0000 7f6673200640 1 -- 172.21.15.82:0/2788956634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666c0ccbe0 con 0x7f666c0d0e70 2024-03-16T17:57:57.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.383+0000 7f6673200640 1 -- 172.21.15.82:0/2788956634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666c0b7bd0 con 0x7f666c0d0e70 2024-03-16T17:57:57.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.384+0000 7f6663e00640 1 -- 172.21.15.82:0/2788956634 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f665c01e460 con 0x7f666c0d0e70 2024-03-16T17:57:57.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.384+0000 7f6663e00640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f664c06b400 0x7f664c06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:57.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.385+0000 7f6663e00640 1 -- 172.21.15.82:0/2788956634 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f665c08a330 con 0x7f666c0d0e70 2024-03-16T17:57:57.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.385+0000 7f666b400640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f664c06b400 0x7f664c06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:57.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.385+0000 7f666b400640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f664c06b400 0x7f664c06d8c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f666c06cb90 tx=0x7f6658009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:57.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.390+0000 7f6663e00640 1 -- 172.21.15.82:0/2788956634 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f665c055000 con 0x7f666c0d0e70 2024-03-16T17:57:57.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.627+0000 7f6673200640 1 -- 172.21.15.82:0/2788956634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f666c0d4bd0 con 0x7f666c0d0e70 2024-03-16T17:57:57.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.628+0000 7f6663e00640 1 -- 172.21.15.82:0/2788956634 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4968 (secure 0 0 0) 0x7f665c0586c0 con 0x7f666c0d0e70 2024-03-16T17:57:57.631 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:57:57.632 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2570794110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2570794110},{"type":"v1","addr":"172.21.15.90:6835","nonce":2570794110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3135739259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3135739259},{"type":"v1","addr":"172.21.15.82:6837","nonce":3135739259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3721205060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3721205060},{"type":"v1","addr":"172.21.15.82:6835","nonce":3721205060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:57:40.797669+0000","modified":"2024-03-16T17:57:43.865338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/732041173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":732041173},{"type":"v1","addr":"172.21.15.90:6833","nonce":732041173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4267,"qdb_cluster":[4267]},"id":1}]} 2024-03-16T17:57:57.632 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-03-16T17:57:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.630+0000 7f6673200640 1 -- 172.21.15.82:0/2788956634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f664c06b400 msgr2=0x7f664c06d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.630+0000 7f6673200640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f664c06b400 0x7f664c06d8c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f666c06cb90 tx=0x7f6658009040 comp rx=0 tx=0).stop 2024-03-16T17:57:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.630+0000 7f6673200640 1 -- 172.21.15.82:0/2788956634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 msgr2=0x7f666c1579e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.630+0000 7f6673200640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 0x7f666c1579e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f665c00d830 tx=0x7f665c00dcf0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.630+0000 7f6673200640 1 -- 172.21.15.82:0/2788956634 shutdown_connections 2024-03-16T17:57:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.630+0000 7f6673200640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f664c06b400 0x7f664c06d8c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.630+0000 7f6673200640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f666c15f110 0x7f666c06c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.630+0000 7f6673200640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f666c15e180 0x7f666c154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.630+0000 7f6673200640 1 --2- 172.21.15.82:0/2788956634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f666c0d0e70 0x7f666c1579e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.630+0000 7f6673200640 1 -- 172.21.15.82:0/2788956634 >> 172.21.15.82:0/2788956634 conn(0x7f666c0d3e90 msgr2=0x7f666c0b74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.631+0000 7f6673200640 1 -- 172.21.15.82:0/2788956634 shutdown_connections 2024-03-16T17:57:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:57.631+0000 7f6673200640 1 -- 172.21.15.82:0/2788956634 wait complete. 2024-03-16T17:57:58.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-16T17:57:40.797669+0000', 'modified': '2024-03-16T17:57:43.865338+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4267}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4267': {'gid': 4267, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.90:6833/732041173', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6832', 'nonce': 732041173}, {'type': 'v1', 'addr': '172.21.15.90:6833', 'nonce': 732041173}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4267, 'qdb_cluster': [4267]} 2024-03-16T17:57:58.054 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-16T17:57:58.054 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-16T17:57:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.273+0000 7f89ef200640 1 Processor -- start 2024-03-16T17:57:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.273+0000 7f89ef200640 1 -- start start 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ef200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f89f01455d0 0x7f89f00cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ef200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89f00b2a20 0x7f89f015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ef200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f89f00cccf0 0x7f89f00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ef200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00cd6a0 con 0x7f89f00cccf0 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ef200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00b3ac0 con 0x7f89f01455d0 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ef200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00b2eb0 con 0x7f89f00b2a20 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ee800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f89f00cccf0 0x7f89f00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ee800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f89f00cccf0 0x7f89f00cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56082/0 (socket says 172.21.15.82:56082) 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ee800640 1 -- 172.21.15.82:0/679661527 learned_addr learned my addr 172.21.15.82:0/679661527 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ed400640 1 --2- 172.21.15.82:0/679661527 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89f00b2a20 0x7f89f015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ede00640 1 --2- 172.21.15.82:0/679661527 >> v2:172.21.15.90:3300/0 conn(0x7f89f01455d0 0x7f89f00cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ee800640 1 -- 172.21.15.82:0/679661527 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89f00b2a20 msgr2=0x7f89f015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ee800640 1 --2- 172.21.15.82:0/679661527 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89f00b2a20 0x7f89f015d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ee800640 1 -- 172.21.15.82:0/679661527 >> v2:172.21.15.90:3300/0 conn(0x7f89f01455d0 msgr2=0x7f89f00cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ee800640 1 --2- 172.21.15.82:0/679661527 >> v2:172.21.15.90:3300/0 conn(0x7f89f01455d0 0x7f89f00cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.275+0000 7f89ee800640 1 -- 172.21.15.82:0/679661527 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00b3190 con 0x7f89f00cccf0 2024-03-16T17:57:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.276+0000 7f89ee800640 1 --2- 172.21.15.82:0/679661527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f00cccf0 0x7f89f00cd160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f89f00731c0 tx=0x7f89e802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba1a755755ef376 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89eca00640 1 -- 172.21.15.82:0/679661527 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e802c9e0 con 0x7f89f00cccf0 2024-03-16T17:57:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89eca00640 1 -- 172.21.15.82:0/679661527 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e802cb40 con 0x7f89f00cccf0 2024-03-16T17:57:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89eca00640 1 -- 172.21.15.82:0/679661527 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80343f0 con 0x7f89f00cccf0 2024-03-16T17:57:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89ef200640 1 -- 172.21.15.82:0/679661527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f00cccf0 msgr2=0x7f89f00cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89ef200640 1 --2- 172.21.15.82:0/679661527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f00cccf0 0x7f89f00cd160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f89f00731c0 tx=0x7f89e802be40 comp rx=0 tx=0).stop 2024-03-16T17:57:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89ef200640 1 -- 172.21.15.82:0/679661527 shutdown_connections 2024-03-16T17:57:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89ef200640 1 --2- 172.21.15.82:0/679661527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f00cccf0 0x7f89f00cd160 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89ef200640 1 --2- 172.21.15.82:0/679661527 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89f00b2a20 0x7f89f015d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89ef200640 1 --2- 172.21.15.82:0/679661527 >> v2:172.21.15.90:3300/0 conn(0x7f89f01455d0 0x7f89f00cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89ef200640 1 -- 172.21.15.82:0/679661527 >> 172.21.15.82:0/679661527 conn(0x7f89f006ae70 msgr2=0x7f89f006b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.277+0000 7f89ef200640 1 -- 172.21.15.82:0/679661527 shutdown_connections 2024-03-16T17:57:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.278+0000 7f89ef200640 1 -- 172.21.15.82:0/679661527 wait complete. 2024-03-16T17:57:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.279+0000 7f89ef200640 1 Processor -- start 2024-03-16T17:57:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.279+0000 7f89ef200640 1 -- start start 2024-03-16T17:57:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.280+0000 7f89ef200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89f00cccf0 0x7f89f0143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.280+0000 7f89ef200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f01455d0 0x7f89f01443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.280+0000 7f89ef200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89f0144930 0x7f89f00c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.280+0000 7f89ef200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0144dc0 con 0x7f89f01455d0 2024-03-16T17:57:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.280+0000 7f89ef200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f0144210 con 0x7f89f00cccf0 2024-03-16T17:57:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.280+0000 7f89ef200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00c9b10 con 0x7f89f0144930 2024-03-16T17:57:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.280+0000 7f89ed400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f01455d0 0x7f89f01443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.280+0000 7f89ed400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f01455d0 0x7f89f01443f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56098/0 (socket says 172.21.15.82:56098) 2024-03-16T17:57:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.280+0000 7f89ed400640 1 -- 172.21.15.82:0/3347435977 learned_addr learned my addr 172.21.15.82:0/3347435977 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89ee800640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89f0144930 0x7f89f00c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89ed400640 1 -- 172.21.15.82:0/3347435977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89f0144930 msgr2=0x7f89f00c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89ed400640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89f0144930 0x7f89f00c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89ed400640 1 -- 172.21.15.82:0/3347435977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89f00cccf0 msgr2=0x7f89f0143cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89ed400640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89f00cccf0 0x7f89f0143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89ed400640 1 -- 172.21.15.82:0/3347435977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e8009b30 con 0x7f89f01455d0 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89ed400640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f01455d0 0x7f89f01443f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00d800 tx=0x7f89dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89e6000640 1 -- 172.21.15.82:0/3347435977 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0096a0 con 0x7f89f01455d0 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89e6000640 1 -- 172.21.15.82:0/3347435977 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc009800 con 0x7f89f01455d0 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89e6000640 1 -- 172.21.15.82:0/3347435977 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc009990 con 0x7f89f01455d0 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.281+0000 7f89ef200640 1 -- 172.21.15.82:0/3347435977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f00c9df0 con 0x7f89f01455d0 2024-03-16T17:57:58.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.282+0000 7f89ef200640 1 -- 172.21.15.82:0/3347435977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f00ca3b0 con 0x7f89f01455d0 2024-03-16T17:57:58.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.284+0000 7f89e6000640 1 -- 172.21.15.82:0/3347435977 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f89dc009af0 con 0x7f89f01455d0 2024-03-16T17:57:58.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.284+0000 7f89e6000640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f89d006b3e0 0x7f89d006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.284+0000 7f89e6000640 1 -- 172.21.15.82:0/3347435977 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89dc089fe0 con 0x7f89f01455d0 2024-03-16T17:57:58.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.285+0000 7f89ede00640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f89d006b3e0 0x7f89d006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.285+0000 7f89ede00640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f89d006b3e0 0x7f89d006d8a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f89e0006fd0 tx=0x7f89e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:58.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.285+0000 7f89f7b89740 1 Processor -- start 2024-03-16T17:57:58.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.285+0000 7f89f7b89740 1 -- start start 2024-03-16T17:57:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.286+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57810 0x563942f2a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.286+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57bf0 0x563942f5a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.286+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f2acf0 0x563942f5aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.286+0000 7f89f7b89740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563942e9eb30 con 0x563942f2acf0 2024-03-16T17:57:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.286+0000 7f89f7b89740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x563942e9c060 con 0x563942f57bf0 2024-03-16T17:57:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.286+0000 7f89f7b89740 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x563942e97840 con 0x563942f57810 2024-03-16T17:57:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.286+0000 7f89ee800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f2acf0 0x563942f5aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89ee800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f2acf0 0x563942f5aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56114/0 (socket says 172.21.15.82:56114) 2024-03-16T17:57:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89ee800640 1 -- 172.21.15.82:0/2573257430 learned_addr learned my addr 172.21.15.82:0/2573257430 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89ede00640 1 --2- 172.21.15.82:0/2573257430 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57810 0x563942f2a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89ee800640 1 -- 172.21.15.82:0/2573257430 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57810 msgr2=0x563942f2a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89ee800640 1 --2- 172.21.15.82:0/2573257430 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57810 0x563942f2a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89ee800640 1 -- 172.21.15.82:0/2573257430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57bf0 msgr2=0x563942f5a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89ee800640 1 --2- 172.21.15.82:0/2573257430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57bf0 0x563942f5a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89ee800640 1 -- 172.21.15.82:0/2573257430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563942e5f4c0 con 0x563942f2acf0 2024-03-16T17:57:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89ede00640 1 --2- 172.21.15.82:0/2573257430 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57810 0x563942f2a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89ee800640 1 --2- 172.21.15.82:0/2573257430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f2acf0 0x563942f5aae0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89e802c450 tx=0x7f89e802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca801e9018afe5d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89daa00640 1 -- 172.21.15.82:0/2573257430 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e802c9e0 con 0x563942f2acf0 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89daa00640 1 -- 172.21.15.82:0/2573257430 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8040470 con 0x563942f2acf0 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.287+0000 7f89daa00640 1 -- 172.21.15.82:0/2573257430 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8040800 con 0x563942f2acf0 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.288+0000 7f89f7b89740 1 -- 172.21.15.82:0/2573257430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f2acf0 msgr2=0x563942f5aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.288+0000 7f89f7b89740 1 --2- 172.21.15.82:0/2573257430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f2acf0 0x563942f5aae0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89e802c450 tx=0x7f89e802e000 comp rx=0 tx=0).stop 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.288+0000 7f89f7b89740 1 -- 172.21.15.82:0/2573257430 shutdown_connections 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.288+0000 7f89f7b89740 1 --2- 172.21.15.82:0/2573257430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f2acf0 0x563942f5aae0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.288+0000 7f89f7b89740 1 --2- 172.21.15.82:0/2573257430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57bf0 0x563942f5a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.288+0000 7f89f7b89740 1 --2- 172.21.15.82:0/2573257430 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57810 0x563942f2a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.288+0000 7f89f7b89740 1 -- 172.21.15.82:0/2573257430 >> 172.21.15.82:0/2573257430 conn(0x563942f2ca20 msgr2=0x563942f755a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.288+0000 7f89f7b89740 1 -- 172.21.15.82:0/2573257430 shutdown_connections 2024-03-16T17:57:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.288+0000 7f89f7b89740 1 -- 172.21.15.82:0/2573257430 wait complete. 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.289+0000 7f89f7b89740 1 -- start start 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.289+0000 7f89f7b89740 1 Processor -- start 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.290+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57810 0x563942f75e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.290+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 0x563942f52470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.290+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f503f0 0x563942f50870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.290+0000 7f89f7b89740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563942e3cea0 con 0x563942f57810 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.290+0000 7f89f7b89740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x563942e3cb40 con 0x563942f503f0 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.290+0000 7f89f7b89740 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x563942bdff20 con 0x563942f57bf0 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.290+0000 7f89f7b89740 10 client.0 _ll_register_callbacks cb 0x563942e50950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.290+0000 7f89ede00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57810 0x563942f75e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.290+0000 7f89ede00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57810 0x563942f75e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56122/0 (socket says 172.21.15.82:56122) 2024-03-16T17:57:58.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.290+0000 7f89ede00640 1 -- 172.21.15.82:0/326059603 learned_addr learned my addr 172.21.15.82:0/326059603 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89ede00640 1 -- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 msgr2=0x563942f52470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89ed400640 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 0x563942f52470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89ede00640 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 0x563942f52470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89ede00640 1 -- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f503f0 msgr2=0x563942f50870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89ee800640 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f503f0 0x563942f50870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89ede00640 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f503f0 0x563942f50870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89ede00640 1 -- 172.21.15.82:0/326059603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f00b3190 con 0x563942f57810 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89ede00640 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57810 0x563942f75e60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f89e000cf50 tx=0x7f89e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486d2a9f4c5effd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89cf400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89cf400640 1 -- 172.21.15.82:0/326059603 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0019050 con 0x563942f57810 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89cf400640 1 -- 172.21.15.82:0/326059603 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89e00117e0 con 0x563942f57810 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89f7b89740 1 -- 172.21.15.82:0/326059603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563942ebbc40 con 0x563942f57810 2024-03-16T17:57:58.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.291+0000 7f89cf400640 1 -- 172.21.15.82:0/326059603 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89e00119a0 con 0x563942f57810 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.292+0000 7f89f7b89740 10 client.4668 fetch_fsmap learned FSMap version 8 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.292+0000 7f89f7b89740 1 -- 172.21.15.82:0/326059603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x563942e95ec0 con 0x563942f57810 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.292+0000 7f89cf400640 1 -- 172.21.15.82:0/326059603 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563942ebbc40 con 0x563942f57810 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.292+0000 7f89cf400640 1 -- 172.21.15.82:0/326059603 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 8) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f89e0011b00 con 0x563942f57810 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.292+0000 7f89f7b89740 10 client.4668 fetch_fsmap finished waiting for FSMap version 8 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.292+0000 7f89f7b89740 10 client.4668 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.292+0000 7f89f7b89740 20 client.4668 populate_metadata read hostname 'smithi082' 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.292+0000 7f89f7b89740 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:6832/732041173,v1:172.21.15.90:6833/732041173] conn(0x563942f896d0 0x563942f58ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.292+0000 7f89f7b89740 4 client.4668 mds_command: new command op to 4267 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.292+0000 7f89f7b89740 1 -- 172.21.15.82:0/326059603 --> [v2:172.21.15.90:6832/732041173,v1:172.21.15.90:6833/732041173] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563942b976c0 con 0x563942f896d0 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.293+0000 7f89ed400640 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:6832/732041173,v1:172.21.15.90:6833/732041173] conn(0x563942f896d0 0x563942f58ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.293+0000 7f89ed400640 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:6832/732041173,v1:172.21.15.90:6833/732041173] conn(0x563942f896d0 0x563942f58ca0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92722f1bc31f53db server_cookie=a04326aab0351e5d in_seq=0 out_seq=0 2024-03-16T17:57:58.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.293+0000 7f89cf400640 10 client.4668 ms_handle_connect on v2:172.21.15.90:6832/732041173 2024-03-16T17:57:58.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89cf400640 1 -- 172.21.15.82:0/326059603 <== mds.0 v2:172.21.15.90:6832/732041173 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563942b976c0 con 0x563942f896d0 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89cf400640 10 client.4668 handle_command_reply: tid=0 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 client.4668 shutdown 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 -- 172.21.15.82:0/326059603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57810 msgr2=0x563942f75e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57810 0x563942f75e60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f89e000cf50 tx=0x7f89e00046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 -- 172.21.15.82:0/326059603 shutdown_connections 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:6832/732041173,v1:172.21.15.90:6833/732041173] conn(0x563942f896d0 0x563942f58ca0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f503f0 0x563942f50870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 0x563942f52470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89cf400640 0 client.4668 ms_handle_reset on v2:172.21.15.90:6832/732041173 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 --2- 172.21.15.82:0/326059603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57810 0x563942f75e60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 -- 172.21.15.82:0/326059603 >> 172.21.15.82:0/326059603 conn(0x563942f2acf0 msgr2=0x563942f2e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 -- 172.21.15.82:0/326059603 shutdown_connections 2024-03-16T17:57:58.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.295+0000 7f89f7b89740 1 -- 172.21.15.82:0/326059603 wait complete. 2024-03-16T17:57:58.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.296+0000 7f89f7b89740 20 client.4668 trim_cache size 0 max 16384 2024-03-16T17:57:58.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.313+0000 7f89f7b89740 1 Processor -- start 2024-03-16T17:57:58.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.313+0000 7f89f7b89740 1 -- start start 2024-03-16T17:57:58.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.314+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f503f0 0x563942f78ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.314+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 0x563942f519e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.314+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 0x563942f5a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.314+0000 7f89f7b89740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563942f59510 con 0x563942f503f0 2024-03-16T17:57:58.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.314+0000 7f89f7b89740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x563942ec8380 con 0x563942f57810 2024-03-16T17:57:58.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.314+0000 7f89f7b89740 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x563942f2bc20 con 0x563942f57bf0 2024-03-16T17:57:58.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ede00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f503f0 0x563942f78ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ed400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 0x563942f519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ede00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f503f0 0x563942f78ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56130/0 (socket says 172.21.15.82:56130) 2024-03-16T17:57:58.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ede00640 1 -- 172.21.15.82:0/1720403092 learned_addr learned my addr 172.21.15.82:0/1720403092 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:58.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ee800640 1 --2- 172.21.15.82:0/1720403092 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 0x563942f5a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ede00640 1 -- 172.21.15.82:0/1720403092 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 msgr2=0x563942f5a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:58.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ede00640 1 --2- 172.21.15.82:0/1720403092 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 0x563942f5a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ede00640 1 -- 172.21.15.82:0/1720403092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 msgr2=0x563942f519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:58.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ede00640 1 --2- 172.21.15.82:0/1720403092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 0x563942f519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ede00640 1 -- 172.21.15.82:0/1720403092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563942e328b0 con 0x563942f503f0 2024-03-16T17:57:58.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ee800640 1 --2- 172.21.15.82:0/1720403092 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 0x563942f5a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:58.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ed400640 1 --2- 172.21.15.82:0/1720403092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 0x563942f519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:57:58.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.315+0000 7f89ede00640 1 --2- 172.21.15.82:0/1720403092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f503f0 0x563942f78ab0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000c00 tx=0x7f89e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73829feb3bc69d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:58.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89da000640 1 -- 172.21.15.82:0/1720403092 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0019040 con 0x563942f503f0 2024-03-16T17:57:58.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89da000640 1 -- 172.21.15.82:0/1720403092 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e001a040 con 0x563942f503f0 2024-03-16T17:57:58.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89da000640 1 -- 172.21.15.82:0/1720403092 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00119e0 con 0x563942f503f0 2024-03-16T17:57:58.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89f7b89740 1 -- 172.21.15.82:0/1720403092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f503f0 msgr2=0x563942f78ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:58.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89f7b89740 1 --2- 172.21.15.82:0/1720403092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f503f0 0x563942f78ab0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000c00 tx=0x7f89e0017000 comp rx=0 tx=0).stop 2024-03-16T17:57:58.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89f7b89740 1 -- 172.21.15.82:0/1720403092 shutdown_connections 2024-03-16T17:57:58.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89f7b89740 1 --2- 172.21.15.82:0/1720403092 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f57bf0 0x563942f5a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89f7b89740 1 --2- 172.21.15.82:0/1720403092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 0x563942f519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89f7b89740 1 --2- 172.21.15.82:0/1720403092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f503f0 0x563942f78ab0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89f7b89740 1 -- 172.21.15.82:0/1720403092 >> 172.21.15.82:0/1720403092 conn(0x563942f2acf0 msgr2=0x563942f52330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:58.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89f7b89740 1 -- 172.21.15.82:0/1720403092 shutdown_connections 2024-03-16T17:57:58.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.316+0000 7f89f7b89740 1 -- 172.21.15.82:0/1720403092 wait complete. 2024-03-16T17:57:58.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.317+0000 7f89f7b89740 1 -- start start 2024-03-16T17:57:58.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.317+0000 7f89f7b89740 1 Processor -- start 2024-03-16T17:57:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.318+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f503f0 0x563942fcd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.318+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 0x563942f58ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.318+0000 7f89f7b89740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57bf0 0x563942f5a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.318+0000 7f89f7b89740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563942fe5d00 con 0x563942f57bf0 2024-03-16T17:57:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.318+0000 7f89f7b89740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x563942fe48e0 con 0x563942f57810 2024-03-16T17:57:58.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.318+0000 7f89f7b89740 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x563942f98150 con 0x563942f503f0 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.318+0000 7f89f7b89740 10 client.0 _ll_register_callbacks cb 0x7f89e000cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.318+0000 7f89ee800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57bf0 0x563942f5a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ee800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57bf0 0x563942f5a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56142/0 (socket says 172.21.15.82:56142) 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ee800640 1 -- 172.21.15.82:0/2795833796 learned_addr learned my addr 172.21.15.82:0/2795833796 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ed400640 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 0x563942f58ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ede00640 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f503f0 0x563942fcd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ee800640 1 -- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f503f0 msgr2=0x563942fcd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ee800640 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f503f0 0x563942fcd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ee800640 1 -- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 msgr2=0x563942f58ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ee800640 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 0x563942f58ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ee800640 1 -- 172.21.15.82:0/2795833796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563942e95ec0 con 0x563942f57bf0 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ed400640 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 0x563942f58ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:57:58.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89ee800640 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57bf0 0x563942f5a740 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f89e8043040 tx=0x7f89e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df7a5372321eb68 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89cf400640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89cf400640 1 -- 172.21.15.82:0/2795833796 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e80407e0 con 0x563942f57bf0 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.319+0000 7f89cf400640 1 -- 172.21.15.82:0/2795833796 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89e8040940 con 0x563942f57bf0 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89f7b89740 1 -- 172.21.15.82:0/2795833796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89e8040470 con 0x563942f57bf0 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89cf400640 1 -- 172.21.15.82:0/2795833796 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89e8040b20 con 0x563942f57bf0 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89f7b89740 10 client.4674 fetch_fsmap learned FSMap version 8 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89f7b89740 1 -- 172.21.15.82:0/2795833796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x563942e58510 con 0x563942f57bf0 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89cf400640 1 -- 172.21.15.82:0/2795833796 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8040470 con 0x563942f57bf0 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89cf400640 1 -- 172.21.15.82:0/2795833796 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 8) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f89e803c040 con 0x563942f57bf0 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89f7b89740 10 client.4674 fetch_fsmap finished waiting for FSMap version 8 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89f7b89740 10 client.4674 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89f7b89740 20 client.4674 populate_metadata read hostname 'smithi082' 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89f7b89740 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:6832/732041173,v1:172.21.15.90:6833/732041173] conn(0x563942f98720 0x563942f98b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:58.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89f7b89740 4 client.4674 mds_command: new command op to 4267 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T17:57:58.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.320+0000 7f89f7b89740 1 -- 172.21.15.82:0/2795833796 --> [v2:172.21.15.90:6832/732041173,v1:172.21.15.90:6833/732041173] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563942be8270 con 0x563942f98720 2024-03-16T17:57:58.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.321+0000 7f89ede00640 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:6832/732041173,v1:172.21.15.90:6833/732041173] conn(0x563942f98720 0x563942f98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:58.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89ede00640 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:6832/732041173,v1:172.21.15.90:6833/732041173] conn(0x563942f98720 0x563942f98b00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6ba9bed8fe9db3 server_cookie=c6bedd531e70cf58 in_seq=0 out_seq=0 2024-03-16T17:57:58.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89cf400640 10 client.4674 ms_handle_connect on v2:172.21.15.90:6832/732041173 2024-03-16T17:57:58.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89cf400640 1 -- 172.21.15.82:0/2795833796 <== mds.0 v2:172.21.15.90:6832/732041173 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x563942be8270 con 0x563942f98720 2024-03-16T17:57:58.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89cf400640 10 client.4674 handle_command_reply: tid=0 2024-03-16T17:57:58.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89f7b89740 1 client.4674 shutdown 2024-03-16T17:57:58.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89f7b89740 1 -- 172.21.15.82:0/2795833796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57bf0 msgr2=0x563942f5a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:58.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89f7b89740 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57bf0 0x563942f5a740 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f89e8043040 tx=0x7f89e8004ab0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89f7b89740 1 -- 172.21.15.82:0/2795833796 shutdown_connections 2024-03-16T17:57:58.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89f7b89740 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:6832/732041173,v1:172.21.15.90:6833/732041173] conn(0x563942f98720 0x563942f98b00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89f7b89740 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563942f57bf0 0x563942f5a740 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89f7b89740 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563942f57810 0x563942f58ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89f7b89740 1 --2- 172.21.15.82:0/2795833796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x563942f503f0 0x563942fcd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89cf400640 0 client.4674 ms_handle_reset on v2:172.21.15.90:6832/732041173 2024-03-16T17:57:58.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.322+0000 7f89f7b89740 1 -- 172.21.15.82:0/2795833796 >> 172.21.15.82:0/2795833796 conn(0x563942f2acf0 msgr2=0x563942f74f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:58.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89f7b89740 1 -- 172.21.15.82:0/2795833796 shutdown_connections 2024-03-16T17:57:58.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89f7b89740 1 -- 172.21.15.82:0/2795833796 wait complete. 2024-03-16T17:57:58.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89f7b89740 20 client.4674 trim_cache size 0 max 16384 2024-03-16T17:57:58.329 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T17:57:58.329 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "08082c2e-06a4-4a16-ab2e-9e9e244a6661", 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 4267, 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 16.959466751000001, 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 8, 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 21, 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 21, 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 24.654685932 2024-03-16T17:57:58.330 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T17:57:58.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 -- 172.21.15.82:0/3347435977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f89d006b3e0 msgr2=0x7f89d006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:58.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f89d006b3e0 0x7f89d006d8a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f89e0006fd0 tx=0x7f89e0008040 comp rx=0 tx=0).stop 2024-03-16T17:57:58.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 -- 172.21.15.82:0/3347435977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f01455d0 msgr2=0x7f89f01443f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:58.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f01455d0 0x7f89f01443f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00d800 tx=0x7f89dc00dcc0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 -- 172.21.15.82:0/3347435977 shutdown_connections 2024-03-16T17:57:58.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f89d006b3e0 0x7f89d006d8a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f89f0144930 0x7f89f00c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89f01455d0 0x7f89f01443f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 --2- 172.21.15.82:0/3347435977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f89f00cccf0 0x7f89f0143cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:58.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 -- 172.21.15.82:0/3347435977 >> 172.21.15.82:0/3347435977 conn(0x7f89f006ae70 msgr2=0x7f89f00b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:58.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 -- 172.21.15.82:0/3347435977 shutdown_connections 2024-03-16T17:57:58.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:58.323+0000 7f89ef200640 1 -- 172.21.15.82:0/3347435977 wait complete. 2024-03-16T17:57:58.343 INFO:teuthology.run_tasks:Running task kclient... 2024-03-16T17:57:58.367 INFO:tasks.kclient:Mounting kernel clients... 2024-03-16T17:57:58.367 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-16T17:57:58.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:57:58.368 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-16T17:57:58.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:57:58.420 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-03-16T17:57:58.510 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-03-16T17:57:58.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T17:57:58.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:57:58.515 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-16T17:57:58.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:57:58.562 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-03-16T17:57:58.700 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-03-16T17:57:58.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T17:57:58.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:57:58.705 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-03-16T17:57:58.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:57:58.763 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-03-16T17:57:58.860 INFO:teuthology.orchestra.run.smithi090.stderr:Cannot find device "ceph-brx" 2024-03-16T17:57:58.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T17:57:58.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:57:58.865 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-03-16T17:57:58.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:57:58.915 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-03-16T17:57:59.006 INFO:teuthology.orchestra.run.smithi090.stderr:Cannot find device "ceph-brx" 2024-03-16T17:57:59.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T17:57:59.010 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-16T17:57:59.010 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T17:57:59.010 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T17:57:59.010 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-16T17:57:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.226+0000 7f1604200640 1 Processor -- start 2024-03-16T17:57:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.226+0000 7f1604200640 1 -- start start 2024-03-16T17:57:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.228+0000 7f1604200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f15fc15e2c0 0x7f15fc0b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.228+0000 7f1604200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15fc15e790 con 0x7f15fc15dee0 2024-03-16T17:57:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.228+0000 7f1604200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15fc15ea70 con 0x7f15fc072a20 2024-03-16T17:57:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.228+0000 7f1604200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15fc15ed50 con 0x7f15fc15e2c0 2024-03-16T17:57:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.228+0000 7f1602200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f15fc15e2c0 0x7f15fc0b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.228+0000 7f1601800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f15fc15dee0 0x7f15fc072310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49122/0 (socket says 172.21.15.82:49122) 2024-03-16T17:57:59.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.228+0000 7f1601800640 1 -- 172.21.15.82:0/1649354400 learned_addr learned my addr 172.21.15.82:0/1649354400 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:57:59.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.229+0000 7f1602200640 1 -- 172.21.15.82:0/1649354400 >> v1:172.21.15.90:6789/0 conn(0x7f15fc072a20 legacy=0x7f15fc0b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:57:59.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.229+0000 7f1602200640 1 -- 172.21.15.82:0/1649354400 >> v1:172.21.15.82:6789/0 conn(0x7f15fc15dee0 legacy=0x7f15fc072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:59.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.229+0000 7f1602200640 1 -- 172.21.15.82:0/1649354400 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc15f030 con 0x7f15fc15e2c0 2024-03-16T17:57:59.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.230+0000 7f1602200640 1 --2- 172.21.15.82:0/1649354400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 0x7f15fc0b48b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15f40097a0 tx=0x7f15f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0af4f92086ec7a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:59.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.230+0000 7f15fbe00640 1 -- 172.21.15.82:0/1649354400 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4031830 con 0x7f15fc15e2c0 2024-03-16T17:57:59.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.230+0000 7f15fbe00640 1 -- 172.21.15.82:0/1649354400 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f4031990 con 0x7f15fc15e2c0 2024-03-16T17:57:59.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.230+0000 7f15fbe00640 1 -- 172.21.15.82:0/1649354400 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4031d20 con 0x7f15fc15e2c0 2024-03-16T17:57:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.231+0000 7f1604200640 1 -- 172.21.15.82:0/1649354400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 msgr2=0x7f15fc0b48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.231+0000 7f1604200640 1 --2- 172.21.15.82:0/1649354400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 0x7f15fc0b48b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f15f40097a0 tx=0x7f15f402b8d0 comp rx=0 tx=0).stop 2024-03-16T17:57:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.231+0000 7f1604200640 1 -- 172.21.15.82:0/1649354400 shutdown_connections 2024-03-16T17:57:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.231+0000 7f1604200640 1 --2- 172.21.15.82:0/1649354400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 0x7f15fc0b48b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.231+0000 7f1604200640 1 -- 172.21.15.82:0/1649354400 >> 172.21.15.82:0/1649354400 conn(0x7f15fc06ae70 msgr2=0x7f15fc06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.231+0000 7f1604200640 1 -- 172.21.15.82:0/1649354400 shutdown_connections 2024-03-16T17:57:59.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.231+0000 7f1604200640 1 -- 172.21.15.82:0/1649354400 wait complete. 2024-03-16T17:57:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.233+0000 7f1604200640 1 Processor -- start 2024-03-16T17:57:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.233+0000 7f1604200640 1 -- start start 2024-03-16T17:57:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.234+0000 7f1604200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f15fc072a20 0x7f15fc148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.234+0000 7f1604200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15fc15dee0 0x7f15fc0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.234+0000 7f1604200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 0x7f15fc0c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.234+0000 7f1604200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc149880 con 0x7f15fc15e2c0 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.234+0000 7f1604200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc1489b0 con 0x7f15fc15dee0 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.234+0000 7f1604200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15fc1486a0 con 0x7f15fc072a20 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1602200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 0x7f15fc0c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1602200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 0x7f15fc0c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56158/0 (socket says 172.21.15.82:56158) 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1602200640 1 -- 172.21.15.82:0/3299734160 learned_addr learned my addr 172.21.15.82:0/3299734160 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1600e00640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15fc15dee0 0x7f15fc0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1601800640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f15fc072a20 0x7f15fc148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1602200640 1 -- 172.21.15.82:0/3299734160 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f15fc072a20 msgr2=0x7f15fc148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1602200640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f15fc072a20 0x7f15fc148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1602200640 1 -- 172.21.15.82:0/3299734160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15fc15dee0 msgr2=0x7f15fc0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1602200640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15fc15dee0 0x7f15fc0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1602200640 1 -- 172.21.15.82:0/3299734160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f4009400 con 0x7f15fc15e2c0 2024-03-16T17:57:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.235+0000 7f1602200640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 0x7f15fc0c4a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15f4034040 tx=0x7f15f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.236+0000 7f15f9600640 1 -- 172.21.15.82:0/3299734160 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f403b9f0 con 0x7f15fc15e2c0 2024-03-16T17:57:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.236+0000 7f15f9600640 1 -- 172.21.15.82:0/3299734160 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f403bb70 con 0x7f15fc15e2c0 2024-03-16T17:57:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.236+0000 7f1604200640 1 -- 172.21.15.82:0/3299734160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15fc148c00 con 0x7f15fc15e2c0 2024-03-16T17:57:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.236+0000 7f15f9600640 1 -- 172.21.15.82:0/3299734160 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f40453f0 con 0x7f15fc15e2c0 2024-03-16T17:57:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.236+0000 7f1604200640 1 -- 172.21.15.82:0/3299734160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15fc1491c0 con 0x7f15fc15e2c0 2024-03-16T17:57:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.237+0000 7f1604200640 1 -- 172.21.15.82:0/3299734160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15fc06cda0 con 0x7f15fc15e2c0 2024-03-16T17:57:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.238+0000 7f15f9600640 1 -- 172.21.15.82:0/3299734160 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f15f4045550 con 0x7f15fc15e2c0 2024-03-16T17:57:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.239+0000 7f15f9600640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f15dc06b3b0 0x7f15dc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:57:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.239+0000 7f15f9600640 1 -- 172.21.15.82:0/3299734160 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15f40abde0 con 0x7f15fc15e2c0 2024-03-16T17:57:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.240+0000 7f1601800640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f15dc06b3b0 0x7f15dc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:57:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.240+0000 7f1601800640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f15dc06b3b0 0x7f15dc06d870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0cfc40 tx=0x7f15f0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:57:59.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.244+0000 7f15f9600640 1 -- 172.21.15.82:0/3299734160 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f15f4076ad0 con 0x7f15fc15e2c0 2024-03-16T17:57:59.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.483+0000 7f1604200640 1 -- 172.21.15.82:0/3299734160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f15fc067f30 con 0x7f15fc15e2c0 2024-03-16T17:57:59.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.484+0000 7f15f9600640 1 -- 172.21.15.82:0/3299734160 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f15f4035020 con 0x7f15fc15e2c0 2024-03-16T17:57:59.487 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T17:57:59.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.486+0000 7f1604200640 1 -- 172.21.15.82:0/3299734160 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f15dc06b3b0 msgr2=0x7f15dc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.486+0000 7f1604200640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f15dc06b3b0 0x7f15dc06d870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0cfc40 tx=0x7f15f0009040 comp rx=0 tx=0).stop 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.486+0000 7f1604200640 1 -- 172.21.15.82:0/3299734160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 msgr2=0x7f15fc0c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.486+0000 7f1604200640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 0x7f15fc0c4a20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f15f4034040 tx=0x7f15f40046c0 comp rx=0 tx=0).stop 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.487+0000 7f1604200640 1 -- 172.21.15.82:0/3299734160 shutdown_connections 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.487+0000 7f1604200640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f15dc06b3b0 0x7f15dc06d870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.487+0000 7f1604200640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15fc15e2c0 0x7f15fc0c4a20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.487+0000 7f1604200640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15fc15dee0 0x7f15fc0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.487+0000 7f1604200640 1 --2- 172.21.15.82:0/3299734160 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f15fc072a20 0x7f15fc148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.487+0000 7f1604200640 1 -- 172.21.15.82:0/3299734160 >> 172.21.15.82:0/3299734160 conn(0x7f15fc06ae70 msgr2=0x7f15fc160960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.487+0000 7f1604200640 1 -- 172.21.15.82:0/3299734160 shutdown_connections 2024-03-16T17:57:59.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:57:59.487+0000 7f1604200640 1 -- 172.21.15.82:0/3299734160 wait complete. 2024-03-16T17:57:59.510 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T17:57:59.510 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T17:57:59.510 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-16T17:57:59.510 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T17:57:59.510 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T17:57:59.510 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T17:57:59.510 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T17:57:59.510 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T17:57:59.510 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T17:57:59.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:57:59.511 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-16T17:57:59.558 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T17:57:59.558 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T17:57:59.558 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T17:57:59.558 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:57:59.558 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-16T17:57:59.558 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:57:59.558 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:57:59.558 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:57:59.558 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42574sec preferred_lft 42574sec 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-16T17:57:59.559 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:57:59.560 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T17:57:59.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:57:59.561 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-16T17:57:59.561 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-16T17:57:59.561 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-03-16T17:57:59.561 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-03-16T17:57:59.561 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-03-16T17:57:59.561 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T17:57:59.561 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-16T17:58:00.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:00.036 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T17:58:00.121 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-16T17:58:00.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:00.125 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-16T17:58:00.176 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-16T17:58:00.176 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T17:58:00.176 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-16T17:58:00.176 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-16T17:58:00.176 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T17:58:00.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:00.177 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-16T17:58:00.177 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-16T17:58:00.177 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-16T17:58:00.177 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-16T17:58:00.177 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-16T17:58:00.177 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-16T17:58:00.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:00.910 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-16T17:58:00.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:00.955 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-03-16T17:58:01.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:01.047 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-16T17:58:01.047 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-16T17:58:01.047 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T17:58:01.047 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-16T17:58:01.047 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-16T17:58:01.175 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-16T17:58:01.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:01.175 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-16T17:58:01.175 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-16T17:58:01.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-16T17:58:01.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T17:58:01.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-16T17:58:01.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-16T17:58:01.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-16T17:58:01.176 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-16T17:58:01.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:01.428 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-16T17:58:01.428 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-16T17:58:01.428 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-03-16T17:58:01.428 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T17:58:01.428 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-16T17:58:01.575 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T17:58:01.575 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-16T17:58:01.617 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-16T17:58:01.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T17:58:01.617 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-16T17:58:01.706 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-16T17:58:01.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:01.706 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T17:58:01.828 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T17:58:01.828 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-16T17:58:01.828 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-03-16T17:58:02.442 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T17:58:02.442 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-16T17:58:02.442 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-03-16T17:58:02.443 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.90:3300,172.21.15.90:3301" 2024-03-16T17:58:02.443 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T17:58:02.483 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T17:58:02.483 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-16T17:58:02.483 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-03-16T17:58:02.483 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.90:3300,172.21.15.90:3301" 2024-03-16T17:58:02.483 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T17:58:02.483 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:6789,172.21.15.90:6789,172.21.15.90:6790" 2024-03-16T17:58:02.483 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.82:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2024-03-16T17:58:02.483 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=08082c2e-06a4-4a16-ab2e-9e9e244a6661" will pass to kernel 2024-03-16T17:58:02.491 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-16T17:58:02.491 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-16T17:58:02.491 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-16T17:58:02.491 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-16T17:58:02.491 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-16T17:58:02.497 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-16T17:58:02.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:02.497 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-16T17:58:02.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:02.590 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:02.670 INFO:teuthology.orchestra.run.smithi082.stdout:client4686 2024-03-16T17:58:02.671 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:58:02.671 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:02.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:02.727 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:02.810 INFO:teuthology.orchestra.run.smithi082.stdout:client4686 2024-03-16T17:58:02.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:02.811 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:02.894 INFO:teuthology.orchestra.run.smithi082.stdout:client4686 2024-03-16T17:58:02.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:02.894 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/08082c2e-06a4-4a16-ab2e-9e9e244a6661.client4686/status') 2024-03-16T17:58:03.007 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.4686 (3)192.168.0.1:0/1277685436 2024-03-16T17:58:03.007 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2024-03-16T17:58:03.007 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2024-03-16T17:58:03.007 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2024-03-16T17:58:03.007 INFO:teuthology.orchestra.run.smithi082.stderr:69 bytes copied, 0.000103334 s, 668 kB/s 2024-03-16T17:58:03.011 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-03-16T17:58:03.011 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T17:58:03.011 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-16T17:58:03.012 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-16T17:58:03.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.218+0000 7f9167e00640 1 Processor -- start 2024-03-16T17:58:03.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.218+0000 7f9167e00640 1 -- start start 2024-03-16T17:58:03.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.220+0000 7f9167e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91680cc520 0x7f91680b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:03.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.220+0000 7f9167e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f91680b4190 0x7f9168070820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:03.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.220+0000 7f9167e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91680b45e0 con 0x7f9168155060 2024-03-16T17:58:03.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.220+0000 7f9167e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f916815f2a0 con 0x7f91680b4190 2024-03-16T17:58:03.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.220+0000 7f9167e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9168072b80 con 0x7f91680cc520 2024-03-16T17:58:03.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.220+0000 7f9166a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9168155060 0x7f9168072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49134/0 (socket says 172.21.15.82:49134) 2024-03-16T17:58:03.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.220+0000 7f9166a00640 1 -- 172.21.15.82:0/1841678044 learned_addr learned my addr 172.21.15.82:0/1841678044 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:03.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.220+0000 7f9167400640 1 --2- 172.21.15.82:0/1841678044 >> v2:172.21.15.90:3300/0 conn(0x7f91680b4190 0x7f9168070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:03.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.220+0000 7f9166000640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91680cc520 0x7f91680b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:03.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416179484 0 0) 0x7f9158001110 con 0x7f9168155060 2024-03-16T17:58:03.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f914c003520 con 0x7f9168155060 2024-03-16T17:58:03.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2656222435 0 0) 0x7f914c003520 con 0x7f9168155060 2024-03-16T17:58:03.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91680cc520 msgr2=0x7f91680b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:03.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91680cc520 0x7f91680b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 >> v2:172.21.15.90:3300/0 conn(0x7f91680b4190 msgr2=0x7f9168070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:03.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 --2- 172.21.15.82:0/1841678044 >> v2:172.21.15.90:3300/0 conn(0x7f91680b4190 0x7f9168070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916815e690 con 0x7f9168155060 2024-03-16T17:58:03.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f9158001800 con 0x7f9168155060 2024-03-16T17:58:03.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 >> v1:172.21.15.82:6789/0 conn(0x7f9168155060 legacy=0x7f9168072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:03.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c0039e0 0x7f914c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:03.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f914c0047c0 0x7f914c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91680cc520 0x7f914c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916815f2a0 con 0x7f914c0039e0 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9168072b80 con 0x7f914c0047c0 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.221+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914c00ba00 con 0x7f91680cc520 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.222+0000 7f9166a00640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c0039e0 0x7f914c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.222+0000 7f9166000640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f914c0047c0 0x7f914c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.222+0000 7f9167400640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91680cc520 0x7f914c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.222+0000 7f9166a00640 1 -- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91680cc520 msgr2=0x7f914c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.222+0000 7f9166a00640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f91680cc520 0x7f914c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.222+0000 7f9166a00640 1 -- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f914c0047c0 msgr2=0x7f914c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.222+0000 7f9166a00640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f914c0047c0 0x7f914c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.222+0000 7f9166a00640 1 -- 172.21.15.82:0/1841678044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914c003520 con 0x7f914c0039e0 2024-03-16T17:58:03.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.224+0000 7f9166a00640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c0039e0 0x7f914c003e00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f915800e420 tx=0x7f914c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5618bcd58c05fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:03.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.224+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158005ad0 con 0x7f914c0039e0 2024-03-16T17:58:03.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.224+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9158031070 con 0x7f914c0039e0 2024-03-16T17:58:03.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.224+0000 7f9165600640 1 -- 172.21.15.82:0/1841678044 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158005e90 con 0x7f914c0039e0 2024-03-16T17:58:03.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.224+0000 7f9167e00640 1 -- 172.21.15.82:0/1841678044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c0039e0 msgr2=0x7f914c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:03.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.224+0000 7f9167e00640 1 --2- 172.21.15.82:0/1841678044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f914c0039e0 0x7f914c003e00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f915800e420 tx=0x7f914c0057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.224+0000 7f9167400640 1 -- 172.21.15.82:0/1841678044 reap_dead start 2024-03-16T17:58:03.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.224+0000 7f9167e00640 1 -- 172.21.15.82:0/1841678044 shutdown_connections 2024-03-16T17:58:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.224+0000 7f9167e00640 1 -- 172.21.15.82:0/1841678044 >> 172.21.15.82:0/1841678044 conn(0x7f91680b1740 msgr2=0x7f91680b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.225+0000 7f9167e00640 1 -- 172.21.15.82:0/1841678044 shutdown_connections 2024-03-16T17:58:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.225+0000 7f9167e00640 1 -- 172.21.15.82:0/1841678044 wait complete. 2024-03-16T17:58:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.226+0000 7f9167e00640 1 Processor -- start 2024-03-16T17:58:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.227+0000 7f9167e00640 1 -- start start 2024-03-16T17:58:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.227+0000 7f9167e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681472f0 0x7f9168153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.227+0000 7f9167e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9168154220 0x7f9168151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.227+0000 7f9167e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9168152270 0x7f9168148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.227+0000 7f9167e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9168153450 con 0x7f91681472f0 2024-03-16T17:58:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.227+0000 7f9167e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9168152ab0 con 0x7f9168152270 2024-03-16T17:58:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.227+0000 7f9167e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9168152d90 con 0x7f9168154220 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9166a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681472f0 0x7f9168153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9166a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681472f0 0x7f9168153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56182/0 (socket says 172.21.15.82:56182) 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9166a00640 1 -- 172.21.15.82:0/510379708 learned_addr learned my addr 172.21.15.82:0/510379708 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9167400640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9168152270 0x7f9168148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9166000640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9168154220 0x7f9168151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9166a00640 1 -- 172.21.15.82:0/510379708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9168154220 msgr2=0x7f9168151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9166a00640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9168154220 0x7f9168151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9166a00640 1 -- 172.21.15.82:0/510379708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9168152270 msgr2=0x7f9168148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9166a00640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9168152270 0x7f9168148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9166a00640 1 -- 172.21.15.82:0/510379708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9168153070 con 0x7f91681472f0 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.228+0000 7f9167400640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9168152270 0x7f9168148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.229+0000 7f9166a00640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681472f0 0x7f9168153ce0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f915800e420 tx=0x7f9158001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.229+0000 7f915ea00640 1 -- 172.21.15.82:0/510379708 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158001850 con 0x7f91681472f0 2024-03-16T17:58:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.229+0000 7f915ea00640 1 -- 172.21.15.82:0/510379708 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915800d070 con 0x7f91681472f0 2024-03-16T17:58:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.229+0000 7f915ea00640 1 -- 172.21.15.82:0/510379708 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9158004ae0 con 0x7f91681472f0 2024-03-16T17:58:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.229+0000 7f9167e00640 1 -- 172.21.15.82:0/510379708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91681546a0 con 0x7f91681472f0 2024-03-16T17:58:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.229+0000 7f9167e00640 1 -- 172.21.15.82:0/510379708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91681476d0 con 0x7f91681472f0 2024-03-16T17:58:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.230+0000 7f9167e00640 1 -- 172.21.15.82:0/510379708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91681478c0 con 0x7f91681472f0 2024-03-16T17:58:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.232+0000 7f915ea00640 1 -- 172.21.15.82:0/510379708 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f9158033030 con 0x7f91681472f0 2024-03-16T17:58:03.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.233+0000 7f915ea00640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f914406b3b0 0x7f914406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.233+0000 7f915ea00640 1 -- 172.21.15.82:0/510379708 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91580aa740 con 0x7f91681472f0 2024-03-16T17:58:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.233+0000 7f9166000640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f914406b3b0 0x7f914406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.234+0000 7f9166000640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f914406b3b0 0x7f914406d870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9168150a60 tx=0x7f9150008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:03.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.238+0000 7f915ea00640 1 -- 172.21.15.82:0/510379708 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9158075430 con 0x7f91681472f0 2024-03-16T17:58:03.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.474+0000 7f9167e00640 1 -- 172.21.15.82:0/510379708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f91680b45e0 con 0x7f91681472f0 2024-03-16T17:58:03.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.474+0000 7f915ea00640 1 -- 172.21.15.82:0/510379708 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f91680b45e0 con 0x7f91681472f0 2024-03-16T17:58:03.478 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 -- 172.21.15.82:0/510379708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f914406b3b0 msgr2=0x7f914406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f914406b3b0 0x7f914406d870 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9168150a60 tx=0x7f9150008040 comp rx=0 tx=0).stop 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 -- 172.21.15.82:0/510379708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681472f0 msgr2=0x7f9168153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681472f0 0x7f9168153ce0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f915800e420 tx=0x7f9158001b90 comp rx=0 tx=0).stop 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 -- 172.21.15.82:0/510379708 shutdown_connections 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f914406b3b0 0x7f914406d870 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9168152270 0x7f9168148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9168154220 0x7f9168151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 --2- 172.21.15.82:0/510379708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91681472f0 0x7f9168153ce0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 -- 172.21.15.82:0/510379708 >> 172.21.15.82:0/510379708 conn(0x7f91680b1740 msgr2=0x7f9168159aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 -- 172.21.15.82:0/510379708 shutdown_connections 2024-03-16T17:58:03.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:03.477+0000 7f9167e00640 1 -- 172.21.15.82:0/510379708 wait complete. 2024-03-16T17:58:03.498 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T17:58:03.499 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T17:58:03.499 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-16T17:58:03.499 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-16T17:58:03.499 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-16T17:58:03.499 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T17:58:03.499 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T17:58:03.499 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T17:58:03.499 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-16T17:58:03.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:03.499 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-16T17:58:03.552 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T17:58:03.552 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T17:58:03.552 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T17:58:03.553 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42570sec preferred_lft 42570sec 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether be:72:9e:97:6e:1f brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::b05f:e0ff:fecd:8301/64 scope link 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether be:72:9e:97:6e:1f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T17:58:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::bc72:9eff:fe97:6e1f/64 scope link 2024-03-16T17:58:03.555 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:03.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:03.556 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T17:58:03.642 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-16T17:58:03.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:03.646 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-16T17:58:03.699 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-16T17:58:03.699 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T17:58:03.699 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-16T17:58:03.699 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-16T17:58:03.699 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T17:58:03.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:03.699 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-16T17:58:03.699 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-16T17:58:03.699 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-16T17:58:03.700 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-16T17:58:03.700 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-16T17:58:03.700 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-16T17:58:03.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:03.878 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-16T17:58:03.926 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-16T17:58:03.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:03.927 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-03-16T17:58:03.976 INFO:teuthology.orchestra.run.smithi082.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-16T17:58:03.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:03.978 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-16T17:58:03.978 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-16T17:58:03.978 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-16T17:58:03.978 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-16T17:58:03.978 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-16T17:58:04.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:04.114 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-16T17:58:04.192 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T17:58:04.192 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T17:58:04.192 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T17:58:04.192 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:04.192 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-16T17:58:04.192 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:04.192 INFO:teuthology.orchestra.run.smithi082.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-16T17:58:04.192 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether a2:04:fa:62:b6:fe brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-16T17:58:04.192 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-16T17:58:04.192 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:04.193 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::a004:faff:fe62:b6fe/64 scope link 2024-03-16T17:58:04.193 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:04.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:04.196 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-16T17:58:04.278 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T17:58:04.278 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T17:58:04.278 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T17:58:04.279 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:04.279 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-16T17:58:04.279 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:04.279 INFO:teuthology.orchestra.run.smithi082.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-16T17:58:04.279 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether a2:04:fa:62:b6:fe brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-16T17:58:04.279 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-16T17:58:04.279 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:04.279 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::a004:faff:fe62:b6fe/64 scope link 2024-03-16T17:58:04.279 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:04.283 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-16T17:58:04.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:04.283 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-16T17:58:04.283 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-16T17:58:04.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-16T17:58:04.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-16T17:58:04.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-16T17:58:04.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-16T17:58:04.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-16T17:58:04.283 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-16T17:58:04.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:04.534 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-16T17:58:04.534 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-16T17:58:04.535 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.1 up 2024-03-16T17:58:04.535 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.1 master ceph-brx 2024-03-16T17:58:04.535 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-16T17:58:04.699 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T17:58:04.699 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-16T17:58:04.745 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-16T17:58:04.745 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T17:58:04.746 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-16T17:58:04.841 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-16T17:58:04.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:04.841 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-16T17:58:04.977 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-16T17:58:04.977 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-16T17:58:04.977 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-03-16T17:58:04.997 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-16T17:58:04.997 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-16T17:58:04.997 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-03-16T17:58:04.997 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.90:3300,172.21.15.90:3301" 2024-03-16T17:58:04.998 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T17:58:05.024 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-16T17:58:05.024 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-16T17:58:05.024 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-03-16T17:58:05.024 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.90:3300,172.21.15.90:3301" 2024-03-16T17:58:05.024 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T17:58:05.024 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:6789,172.21.15.90:6789,172.21.15.90:6790" 2024-03-16T17:58:05.024 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.82:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2024-03-16T17:58:05.024 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=08082c2e-06a4-4a16-ab2e-9e9e244a6661" will pass to kernel 2024-03-16T17:58:05.027 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-16T17:58:05.027 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-16T17:58:05.027 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-16T17:58:05.027 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-16T17:58:05.027 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-16T17:58:05.032 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-16T17:58:05.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:05.033 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-16T17:58:05.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:05.118 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:05.175 INFO:teuthology.orchestra.run.smithi082.stdout:client4695 2024-03-16T17:58:05.176 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:58:05.176 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:05.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:05.229 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:05.284 INFO:teuthology.orchestra.run.smithi082.stdout:client4695 2024-03-16T17:58:05.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:05.284 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:05.340 INFO:teuthology.orchestra.run.smithi082.stdout:client4695 2024-03-16T17:58:05.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:05.340 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/08082c2e-06a4-4a16-ab2e-9e9e244a6661.client4695/status') 2024-03-16T17:58:05.347 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:58:05.393 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:58:05.431 INFO:teuthology.orchestra.run.smithi082.stdout:instance: client.4695 (3)192.168.0.2:0/3913340592 2024-03-16T17:58:05.431 INFO:teuthology.orchestra.run.smithi082.stdout:blocklisted: false 2024-03-16T17:58:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records in 2024-03-16T17:58:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:0+1 records out 2024-03-16T17:58:05.431 INFO:teuthology.orchestra.run.smithi082.stderr:69 bytes copied, 5.4841e-05 s, 1.3 MB/s 2024-03-16T17:58:05.434 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-16T17:58:05.434 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T17:58:05.434 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-16T17:58:05.434 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-16T17:58:05.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.668+0000 7f19a2400640 1 Processor -- start 2024-03-16T17:58:05.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.668+0000 7f19a2400640 1 -- start start 2024-03-16T17:58:05.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.669+0000 7f19a2400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c0d0040 0x7f199c166c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:05.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.669+0000 7f19a2400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c0b4b90 con 0x7f199c0cdf80 2024-03-16T17:58:05.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.669+0000 7f19a2400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199c0cf500 con 0x7f199c1454d0 2024-03-16T17:58:05.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.669+0000 7f19a2400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c1458b0 con 0x7f199c0d0040 2024-03-16T17:58:05.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.669+0000 7f1992a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f199c0cdf80 0x7f199c0cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51250/0 (socket says 172.21.15.82:51250) 2024-03-16T17:58:05.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.669+0000 7f1992a00640 1 -- 172.21.15.82:0/532845952 learned_addr learned my addr 172.21.15.82:0/532845952 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:05.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199be00640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c0d0040 0x7f199c166c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:05.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344417072 0 0) 0x7f199c0b4b90 con 0x7f199c0cdf80 2024-03-16T17:58:05.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1980003520 con 0x7f199c0cdf80 2024-03-16T17:58:05.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2274577995 0 0) 0x7f1980003520 con 0x7f199c0cdf80 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c0d0040 msgr2=0x7f199c166c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c0d0040 0x7f199c166c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 >> v1:172.21.15.90:6789/0 conn(0x7f199c1454d0 legacy=0x7f199c0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c0ce460 con 0x7f199c0cdf80 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f1984001670 con 0x7f199c0cdf80 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 >> v1:172.21.15.82:6789/0 conn(0x7f199c0cdf80 legacy=0x7f199c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19800039a0 0x7f1980003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 0x7f1980004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c0d0040 0x7f1980005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c1458b0 con 0x7f19800047c0 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198000ba60 con 0x7f19800039a0 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.670+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198000bc20 con 0x7f199c0d0040 2024-03-16T17:58:05.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.671+0000 7f1992a00640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 0x7f1980004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:05.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.671+0000 7f199b400640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19800039a0 0x7f1980003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:05.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.671+0000 7f199be00640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c0d0040 0x7f1980005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:05.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.671+0000 7f1992a00640 1 -- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c0d0040 msgr2=0x7f1980005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:05.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.671+0000 7f1992a00640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c0d0040 0x7f1980005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:05.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.671+0000 7f1992a00640 1 -- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19800039a0 msgr2=0x7f1980003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:05.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.671+0000 7f1992a00640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f19800039a0 0x7f1980003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:05.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.671+0000 7f1992a00640 1 -- 172.21.15.82:0/532845952 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980003520 con 0x7f19800047c0 2024-03-16T17:58:05.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.672+0000 7f1992a00640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 0x7f1980004c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f199c0ce390 tx=0x7f19800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147a04535b7c514b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:05.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.672+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984005ad0 con 0x7f19800047c0 2024-03-16T17:58:05.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.672+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1984005c30 con 0x7f19800047c0 2024-03-16T17:58:05.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.672+0000 7f199aa00640 1 -- 172.21.15.82:0/532845952 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198402c850 con 0x7f19800047c0 2024-03-16T17:58:05.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.673+0000 7f19a2400640 1 -- 172.21.15.82:0/532845952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 msgr2=0x7f1980004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:05.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.673+0000 7f19a2400640 1 --2- 172.21.15.82:0/532845952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 0x7f1980004c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f199c0ce390 tx=0x7f19800057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:05.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.673+0000 7f199be00640 1 -- 172.21.15.82:0/532845952 reap_dead start 2024-03-16T17:58:05.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.673+0000 7f19a2400640 1 -- 172.21.15.82:0/532845952 shutdown_connections 2024-03-16T17:58:05.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.673+0000 7f19a2400640 1 -- 172.21.15.82:0/532845952 >> 172.21.15.82:0/532845952 conn(0x7f199c06ae70 msgr2=0x7f199c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:05.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.673+0000 7f19a2400640 1 -- 172.21.15.82:0/532845952 shutdown_connections 2024-03-16T17:58:05.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.673+0000 7f19a2400640 1 -- 172.21.15.82:0/532845952 wait complete. 2024-03-16T17:58:05.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.675+0000 7f19a2400640 1 Processor -- start 2024-03-16T17:58:05.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.675+0000 7f19a2400640 1 -- start start 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f19a2400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 0x7f199c13b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f19a2400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c15eff0 0x7f199c1548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f19a2400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f199c154e10 0x7f199c152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f19a2400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c155290 con 0x7f19800047c0 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f19a2400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199c13b990 con 0x7f199c154e10 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f19a2400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199c1f5970 con 0x7f199c15eff0 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f199b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 0x7f199c13b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f199b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 0x7f199c13b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40192/0 (socket says 172.21.15.82:40192) 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f199b400640 1 -- 172.21.15.82:0/3715293954 learned_addr learned my addr 172.21.15.82:0/3715293954 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f199b400640 1 -- 172.21.15.82:0/3715293954 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c15eff0 msgr2=0x7f199c1548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f1992a00640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c15eff0 0x7f199c1548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:05.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f199be00640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f199c154e10 0x7f199c152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f199b400640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c15eff0 0x7f199c1548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f199b400640 1 -- 172.21.15.82:0/3715293954 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f199c154e10 msgr2=0x7f199c152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.676+0000 7f199b400640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f199c154e10 0x7f199c152d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.677+0000 7f199b400640 1 -- 172.21.15.82:0/3715293954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199c1f4a80 con 0x7f19800047c0 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.677+0000 7f1992a00640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c15eff0 0x7f199c1548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.677+0000 7f199b400640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 0x7f199c13b450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e4d0 tx=0x7f198c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.677+0000 7f1993e00640 1 -- 172.21.15.82:0/3715293954 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c0052e0 con 0x7f19800047c0 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.677+0000 7f1993e00640 1 -- 172.21.15.82:0/3715293954 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198c005440 con 0x7f19800047c0 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.677+0000 7f19a2400640 1 -- 172.21.15.82:0/3715293954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199c1f4d60 con 0x7f19800047c0 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.677+0000 7f1993e00640 1 -- 172.21.15.82:0/3715293954 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c003810 con 0x7f19800047c0 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.677+0000 7f19a2400640 1 -- 172.21.15.82:0/3715293954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c1f5320 con 0x7f19800047c0 2024-03-16T17:58:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.678+0000 7f19a2400640 1 -- 172.21.15.82:0/3715293954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199c067f30 con 0x7f19800047c0 2024-03-16T17:58:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.679+0000 7f1993e00640 1 -- 172.21.15.82:0/3715293954 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f198c003970 con 0x7f19800047c0 2024-03-16T17:58:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.680+0000 7f1993e00640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f197806b3b0 0x7f197806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.680+0000 7f1993e00640 1 -- 172.21.15.82:0/3715293954 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f198c089a00 con 0x7f19800047c0 2024-03-16T17:58:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.680+0000 7f1992a00640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f197806b3b0 0x7f197806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.681+0000 7f1992a00640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f197806b3b0 0x7f197806d870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1984034000 tx=0x7f1984009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:05.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.685+0000 7f1993e00640 1 -- 172.21.15.82:0/3715293954 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f198c0545c0 con 0x7f19800047c0 2024-03-16T17:58:05.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.928+0000 7f19a2400640 1 -- 172.21.15.82:0/3715293954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f199c0b4640 con 0x7f19800047c0 2024-03-16T17:58:05.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.929+0000 7f1993e00640 1 -- 172.21.15.82:0/3715293954 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f198c057c80 con 0x7f19800047c0 2024-03-16T17:58:05.933 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T17:58:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 -- 172.21.15.82:0/3715293954 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f197806b3b0 msgr2=0x7f197806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f197806b3b0 0x7f197806d870 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1984034000 tx=0x7f1984009030 comp rx=0 tx=0).stop 2024-03-16T17:58:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 -- 172.21.15.82:0/3715293954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 msgr2=0x7f199c13b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 0x7f199c13b450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e4d0 tx=0x7f198c00e990 comp rx=0 tx=0).stop 2024-03-16T17:58:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 -- 172.21.15.82:0/3715293954 shutdown_connections 2024-03-16T17:58:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f197806b3b0 0x7f197806d870 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f199c154e10 0x7f199c152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f199c15eff0 0x7f199c1548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:05.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 --2- 172.21.15.82:0/3715293954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19800047c0 0x7f199c13b450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:05.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 -- 172.21.15.82:0/3715293954 >> 172.21.15.82:0/3715293954 conn(0x7f199c06ae70 msgr2=0x7f199c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:05.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 -- 172.21.15.82:0/3715293954 shutdown_connections 2024-03-16T17:58:05.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:05.932+0000 7f19a2400640 1 -- 172.21.15.82:0/3715293954 wait complete. 2024-03-16T17:58:05.953 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T17:58:05.953 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T17:58:05.953 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-03-16T17:58:05.953 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-16T17:58:05.953 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-16T17:58:05.954 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T17:58:05.954 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T17:58:05.954 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T17:58:05.954 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-16T17:58:05.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:05.954 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-03-16T17:58:06.010 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T17:58:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T17:58:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T17:58:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-03-16T17:58:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:06.010 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 42585sec preferred_lft 42585sec 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-03-16T17:58:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:06.012 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T17:58:06.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:06.013 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-16T17:58:06.013 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-16T17:58:06.013 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-03-16T17:58:06.013 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-03-16T17:58:06.013 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-03-16T17:58:06.013 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T17:58:06.013 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-16T17:58:06.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:06.505 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T17:58:06.588 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-03-16T17:58:06.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:06.592 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-03-16T17:58:06.655 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-03-16T17:58:06.655 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T17:58:06.655 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-16T17:58:06.655 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-16T17:58:06.655 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T17:58:06.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:06.656 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-16T17:58:06.656 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-16T17:58:06.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-16T17:58:06.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-16T17:58:06.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-16T17:58:06.657 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-16T17:58:07.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:07.431 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-03-16T17:58:07.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:07.486 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-03-16T17:58:07.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:07.535 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-16T17:58:07.535 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-16T17:58:07.536 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-16T17:58:07.536 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-16T17:58:07.536 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-16T17:58:07.670 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-16T17:58:07.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:07.671 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-16T17:58:07.671 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-16T17:58:07.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-16T17:58:07.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T17:58:07.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-16T17:58:07.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-16T17:58:07.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-16T17:58:07.671 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-16T17:58:07.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:07.952 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-16T17:58:07.952 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-16T17:58:07.952 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-03-16T17:58:07.952 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T17:58:07.952 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-16T17:58:08.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T17:58:08.106 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-16T17:58:08.149 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-16T17:58:08.149 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T17:58:08.150 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-16T17:58:08.240 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-16T17:58:08.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:08.240 DEBUG:teuthology.orchestra.run.smithi090:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-16T17:58:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-16T17:58:08.401 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-16T17:58:08.402 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-03-16T17:58:09.138 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-16T17:58:09.138 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-16T17:58:09.138 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-03-16T17:58:09.138 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.90:3300,172.21.15.90:3301" 2024-03-16T17:58:09.138 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T17:58:09.175 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-16T17:58:09.175 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-16T17:58:09.175 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-03-16T17:58:09.175 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.90:3300,172.21.15.90:3301" 2024-03-16T17:58:09.175 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T17:58:09.175 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.82:6789,172.21.15.90:6789,172.21.15.90:6790" 2024-03-16T17:58:09.175 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.82:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2024-03-16T17:58:09.175 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=08082c2e-06a4-4a16-ab2e-9e9e244a6661" will pass to kernel 2024-03-16T17:58:09.183 INFO:teuthology.orchestra.run.smithi090.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-16T17:58:09.184 INFO:teuthology.orchestra.run.smithi090.stdout: You just mounted a file system that supports labels which does not 2024-03-16T17:58:09.184 INFO:teuthology.orchestra.run.smithi090.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-16T17:58:09.184 INFO:teuthology.orchestra.run.smithi090.stdout: applications will generate AVC messages and not be allowed access to 2024-03-16T17:58:09.184 INFO:teuthology.orchestra.run.smithi090.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-16T17:58:09.188 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-16T17:58:09.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:09.188 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-16T17:58:09.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:09.275 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:09.330 INFO:teuthology.orchestra.run.smithi090.stdout:client4704 2024-03-16T17:58:09.330 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:58:09.330 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:09.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:09.374 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:09.470 INFO:teuthology.orchestra.run.smithi090.stdout:client4704 2024-03-16T17:58:09.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:09.471 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:09.516 INFO:teuthology.orchestra.run.smithi090.stdout:client4704 2024-03-16T17:58:09.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:09.517 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/08082c2e-06a4-4a16-ab2e-9e9e244a6661.client4704/status') 2024-03-16T17:58:09.648 INFO:teuthology.orchestra.run.smithi090.stdout:instance: client.4704 (3)172.21.15.90:0/1211786595 2024-03-16T17:58:09.648 INFO:teuthology.orchestra.run.smithi090.stdout:blocklisted: false 2024-03-16T17:58:09.649 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records in 2024-03-16T17:58:09.649 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records out 2024-03-16T17:58:09.649 INFO:teuthology.orchestra.run.smithi090.stderr:70 bytes copied, 6.1995e-05 s, 1.1 MB/s 2024-03-16T17:58:09.652 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-03-16T17:58:09.652 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T17:58:09.652 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-16T17:58:09.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-16T17:58:09.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.856+0000 7fb5c6200640 1 Processor -- start 2024-03-16T17:58:09.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.856+0000 7fb5c6200640 1 -- start start 2024-03-16T17:58:09.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.857+0000 7fb5c6200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb5c0144400 0x7fb5c00cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:09.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.857+0000 7fb5c6200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb5c015f550 0x7fb5c00cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:09.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.857+0000 7fb5c6200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c015e820 con 0x7fb5c015b310 2024-03-16T17:58:09.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.857+0000 7fb5c6200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c015f130 con 0x7fb5c015f550 2024-03-16T17:58:09.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.857+0000 7fb5c6200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c015be90 con 0x7fb5c0144400 2024-03-16T17:58:09.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.858+0000 7fb5b7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb5c015f550 0x7fb5c00cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.858+0000 7fb5b7e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb5c015f550 0x7fb5c00cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40196/0 (socket says 172.21.15.82:40196) 2024-03-16T17:58:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.858+0000 7fb5b7e00640 1 -- 172.21.15.82:0/2667166076 learned_addr learned my addr 172.21.15.82:0/2667166076 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.858+0000 7fb5bf400640 1 --2- 172.21.15.82:0/2667166076 >> v2:172.21.15.90:3300/0 conn(0x7fb5c0144400 0x7fb5c00cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.858+0000 7fb5b7e00640 1 -- 172.21.15.82:0/2667166076 >> v2:172.21.15.90:3300/0 conn(0x7fb5c0144400 msgr2=0x7fb5c00cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.858+0000 7fb5b7e00640 1 --2- 172.21.15.82:0/2667166076 >> v2:172.21.15.90:3300/0 conn(0x7fb5c0144400 0x7fb5c00cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.858+0000 7fb5b7e00640 1 -- 172.21.15.82:0/2667166076 >> v1:172.21.15.90:6789/0 conn(0x7fb5c015b310 legacy=0x7fb5c015b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.859+0000 7fb5b7e00640 1 -- 172.21.15.82:0/2667166076 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c015c080 con 0x7fb5c015f550 2024-03-16T17:58:09.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.860+0000 7fb5b7e00640 1 --2- 172.21.15.82:0/2667166076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015f550 0x7fb5c00cfed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb5c006cd70 tx=0x7fb5a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ee4d7f928d0bb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.860+0000 7fb5bea00640 1 -- 172.21.15.82:0/2667166076 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8031870 con 0x7fb5c015f550 2024-03-16T17:58:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.860+0000 7fb5bea00640 1 -- 172.21.15.82:0/2667166076 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a80319d0 con 0x7fb5c015f550 2024-03-16T17:58:09.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.860+0000 7fb5bea00640 1 -- 172.21.15.82:0/2667166076 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8031d60 con 0x7fb5c015f550 2024-03-16T17:58:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.861+0000 7fb5c6200640 1 -- 172.21.15.82:0/2667166076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015f550 msgr2=0x7fb5c00cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.861+0000 7fb5c6200640 1 --2- 172.21.15.82:0/2667166076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015f550 0x7fb5c00cfed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb5c006cd70 tx=0x7fb5a802b910 comp rx=0 tx=0).stop 2024-03-16T17:58:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.861+0000 7fb5c6200640 1 -- 172.21.15.82:0/2667166076 shutdown_connections 2024-03-16T17:58:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.861+0000 7fb5c6200640 1 --2- 172.21.15.82:0/2667166076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015f550 0x7fb5c00cfed0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.861+0000 7fb5c6200640 1 --2- 172.21.15.82:0/2667166076 >> v2:172.21.15.90:3300/0 conn(0x7fb5c0144400 0x7fb5c00cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.861+0000 7fb5c6200640 1 -- 172.21.15.82:0/2667166076 >> 172.21.15.82:0/2667166076 conn(0x7fb5c006ae70 msgr2=0x7fb5c006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.861+0000 7fb5c6200640 1 -- 172.21.15.82:0/2667166076 shutdown_connections 2024-03-16T17:58:09.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.861+0000 7fb5c6200640 1 -- 172.21.15.82:0/2667166076 wait complete. 2024-03-16T17:58:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.863+0000 7fb5c6200640 1 Processor -- start 2024-03-16T17:58:09.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.863+0000 7fb5c6200640 1 -- start start 2024-03-16T17:58:09.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.864+0000 7fb5c6200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5c0144400 0x7fb5c01f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:09.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.864+0000 7fb5c6200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015b310 0x7fb5c00aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:09.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.864+0000 7fb5c6200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5c015f550 0x7fb5c00aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:09.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.864+0000 7fb5c6200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c01e5730 con 0x7fb5c015b310 2024-03-16T17:58:09.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.864+0000 7fb5c6200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c01f5dd0 con 0x7fb5c0144400 2024-03-16T17:58:09.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.864+0000 7fb5c6200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c00a9a60 con 0x7fb5c015f550 2024-03-16T17:58:09.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.864+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015b310 0x7fb5c00aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:09.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.864+0000 7fb5b7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015b310 0x7fb5c00aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40212/0 (socket says 172.21.15.82:40212) 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.864+0000 7fb5b7e00640 1 -- 172.21.15.82:0/341020206 learned_addr learned my addr 172.21.15.82:0/341020206 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5bf400640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5c0144400 0x7fb5c01f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5bfe00640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5c015f550 0x7fb5c00aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5b7e00640 1 -- 172.21.15.82:0/341020206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5c015f550 msgr2=0x7fb5c00aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5b7e00640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5c015f550 0x7fb5c00aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5b7e00640 1 -- 172.21.15.82:0/341020206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5c0144400 msgr2=0x7fb5c01f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5b7e00640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5c0144400 0x7fb5c01f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5b7e00640 1 -- 172.21.15.82:0/341020206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a80094d0 con 0x7fb5c015b310 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5b7e00640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015b310 0x7fb5c00aea80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb5a802b950 tx=0x7fb5a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5b7400640 1 -- 172.21.15.82:0/341020206 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a803ad30 con 0x7fb5c015b310 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5b7400640 1 -- 172.21.15.82:0/341020206 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a80443f0 con 0x7fb5c015b310 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5b7400640 1 -- 172.21.15.82:0/341020206 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80447a0 con 0x7fb5c015b310 2024-03-16T17:58:09.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.865+0000 7fb5c6200640 1 -- 172.21.15.82:0/341020206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c01e4bb0 con 0x7fb5c015b310 2024-03-16T17:58:09.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.866+0000 7fb5c6200640 1 -- 172.21.15.82:0/341020206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c01e5170 con 0x7fb5c015b310 2024-03-16T17:58:09.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.867+0000 7fb5c6200640 1 -- 172.21.15.82:0/341020206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c0067f30 con 0x7fb5c015b310 2024-03-16T17:58:09.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.867+0000 7fb5b7400640 1 -- 172.21.15.82:0/341020206 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fb5a804d450 con 0x7fb5c015b310 2024-03-16T17:58:09.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.868+0000 7fb5b7400640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb59c06b3e0 0x7fb59c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:09.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.868+0000 7fb5b7400640 1 -- 172.21.15.82:0/341020206 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5a802d040 con 0x7fb5c015b310 2024-03-16T17:58:09.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.869+0000 7fb5bf400640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb59c06b3e0 0x7fb59c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:09.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.869+0000 7fb5bf400640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb59c06b3e0 0x7fb59c06d8a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0006fd0 tx=0x7fb5b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:09.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:09.873+0000 7fb5b7400640 1 -- 172.21.15.82:0/341020206 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb5a8076ec0 con 0x7fb5c015b310 2024-03-16T17:58:10.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.110+0000 7fb5c6200640 1 -- 172.21.15.82:0/341020206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5c015d1f0 con 0x7fb5c015b310 2024-03-16T17:58:10.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.111+0000 7fb5b7400640 1 -- 172.21.15.82:0/341020206 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb5a8035070 con 0x7fb5c015b310 2024-03-16T17:58:10.115 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T17:58:10.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.114+0000 7fb5c6200640 1 -- 172.21.15.82:0/341020206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb59c06b3e0 msgr2=0x7fb59c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:10.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.114+0000 7fb5c6200640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb59c06b3e0 0x7fb59c06d8a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0006fd0 tx=0x7fb5b0008040 comp rx=0 tx=0).stop 2024-03-16T17:58:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.114+0000 7fb5c6200640 1 -- 172.21.15.82:0/341020206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015b310 msgr2=0x7fb5c00aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.114+0000 7fb5c6200640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015b310 0x7fb5c00aea80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb5a802b950 tx=0x7fb5a80046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.114+0000 7fb5c6200640 1 -- 172.21.15.82:0/341020206 shutdown_connections 2024-03-16T17:58:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.114+0000 7fb5c6200640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb59c06b3e0 0x7fb59c06d8a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.114+0000 7fb5c6200640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb5c015f550 0x7fb5c00aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.114+0000 7fb5c6200640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5c015b310 0x7fb5c00aea80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.114+0000 7fb5c6200640 1 --2- 172.21.15.82:0/341020206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb5c0144400 0x7fb5c01f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.114+0000 7fb5c6200640 1 -- 172.21.15.82:0/341020206 >> 172.21.15.82:0/341020206 conn(0x7fb5c006ae70 msgr2=0x7fb5c006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.115+0000 7fb5c6200640 1 -- 172.21.15.82:0/341020206 shutdown_connections 2024-03-16T17:58:10.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:10.115+0000 7fb5c6200640 1 -- 172.21.15.82:0/341020206 wait complete. 2024-03-16T17:58:10.137 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T17:58:10.138 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T17:58:10.138 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-03-16T17:58:10.138 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-16T17:58:10.138 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-16T17:58:10.138 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T17:58:10.138 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T17:58:10.138 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T17:58:10.138 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-16T17:58:10.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:10.138 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-03-16T17:58:10.189 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T17:58:10.189 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T17:58:10.189 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T17:58:10.189 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.189 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-03-16T17:58:10.189 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 42581sec preferred_lft 42581sec 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::ec4:7aff:fe8f:d767/64 scope link 2024-03-16T17:58:10.190 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run.smithi090.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether fa:d9:03:e7:2c:c6 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run.smithi090.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::54e9:7dff:fe8c:e80e/64 scope link 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run.smithi090.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether fa:d9:03:e7:2c:c6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::f8d9:3ff:fee7:2cc6/64 scope link 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:10.191 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T17:58:10.262 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-03-16T17:58:10.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:10.266 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-03-16T17:58:10.308 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-03-16T17:58:10.308 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T17:58:10.308 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-16T17:58:10.308 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-16T17:58:10.308 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T17:58:10.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:10.308 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-16T17:58:10.308 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-16T17:58:10.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-16T17:58:10.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-16T17:58:10.309 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-16T17:58:10.309 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-16T17:58:10.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:10.522 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-03-16T17:58:10.570 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-16T17:58:10.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:10.571 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-03-16T17:58:10.618 INFO:teuthology.orchestra.run.smithi090.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-16T17:58:10.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:10.619 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-16T17:58:10.619 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-16T17:58:10.619 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-16T17:58:10.619 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-16T17:58:10.619 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-16T17:58:10.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:10.750 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-16T17:58:10.832 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T17:58:10.832 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T17:58:10.832 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T17:58:10.833 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.833 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-03-16T17:58:10.833 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.833 INFO:teuthology.orchestra.run.smithi090.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-16T17:58:10.833 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether ca:2c:dc:a5:b8:79 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-16T17:58:10.833 INFO:teuthology.orchestra.run.smithi090.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-16T17:58:10.834 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.834 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::c82c:dcff:fea5:b879/64 scope link 2024-03-16T17:58:10.834 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:10.836 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether ca:2c:dc:a5:b8:79 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.928 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::c82c:dcff:fea5:b879/64 scope link 2024-03-16T17:58:10.929 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:10.932 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-16T17:58:10.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:10.932 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-16T17:58:10.932 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-16T17:58:10.932 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-16T17:58:10.933 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-16T17:58:10.933 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-16T17:58:10.933 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-16T17:58:10.933 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-16T17:58:10.933 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-16T17:58:11.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:11.196 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec bash -c ' 2024-03-16T17:58:11.196 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-03-16T17:58:11.196 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.1 up 2024-03-16T17:58:11.196 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.1 master ceph-brx 2024-03-16T17:58:11.196 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-03-16T17:58:11.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T17:58:11.350 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-16T17:58:11.393 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-16T17:58:11.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T17:58:11.393 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-16T17:58:11.482 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-16T17:58:11.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:11.483 DEBUG:teuthology.orchestra.run.smithi090:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-16T17:58:11.612 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-16T17:58:11.612 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-16T17:58:11.612 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-03-16T17:58:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-16T17:58:11.634 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-16T17:58:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-03-16T17:58:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.90:3300,172.21.15.90:3301" 2024-03-16T17:58:11.635 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T17:58:11.659 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-16T17:58:11.659 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-16T17:58:11.659 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-03-16T17:58:11.659 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.90:3300,172.21.15.90:3301" 2024-03-16T17:58:11.659 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T17:58:11.659 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.82:6789,172.21.15.90:6789,172.21.15.90:6790" 2024-03-16T17:58:11.659 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.82:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2024-03-16T17:58:11.659 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=08082c2e-06a4-4a16-ab2e-9e9e244a6661" will pass to kernel 2024-03-16T17:58:11.660 INFO:teuthology.orchestra.run.smithi090.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-16T17:58:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: You just mounted a file system that supports labels which does not 2024-03-16T17:58:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-16T17:58:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: applications will generate AVC messages and not be allowed access to 2024-03-16T17:58:11.661 INFO:teuthology.orchestra.run.smithi090.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-16T17:58:11.666 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-16T17:58:11.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:11.666 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-16T17:58:11.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:11.761 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:11.809 INFO:teuthology.orchestra.run.smithi090.stdout:client4412 2024-03-16T17:58:11.809 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:58:11.809 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:11.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:11.901 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:11.994 INFO:teuthology.orchestra.run.smithi090.stdout:client4412 2024-03-16T17:58:11.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:11.998 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:12.052 INFO:teuthology.orchestra.run.smithi090.stdout:client4412 2024-03-16T17:58:12.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:12.053 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/08082c2e-06a4-4a16-ab2e-9e9e244a6661.client4412/status') 2024-03-16T17:58:12.139 INFO:teuthology.orchestra.run.smithi090.stdout:instance: client.4412 (3)192.168.0.2:0/3023863283 2024-03-16T17:58:12.140 INFO:teuthology.orchestra.run.smithi090.stdout:blocklisted: false 2024-03-16T17:58:12.140 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records in 2024-03-16T17:58:12.140 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records out 2024-03-16T17:58:12.140 INFO:teuthology.orchestra.run.smithi090.stderr:69 bytes copied, 8.5272e-05 s, 809 kB/s 2024-03-16T17:58:12.144 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-16T17:58:12.160 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:12.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.378+0000 7f521b200640 1 Processor -- start 2024-03-16T17:58:12.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.378+0000 7f521b200640 1 -- start start 2024-03-16T17:58:12.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.380+0000 7f521b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f521c15e060 0x7f521c0d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.380+0000 7f521b200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f521c0b2bb0 0x7f521c0d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.380+0000 7f521b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521c0d1de0 con 0x7f521c0d27a0 2024-03-16T17:58:12.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.380+0000 7f521b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c0d2c30 con 0x7f521c15e060 2024-03-16T17:58:12.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.380+0000 7f521b200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0d1b00 con 0x7f521c0b2bb0 2024-03-16T17:58:12.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.380+0000 7f5219e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f521c15e060 0x7f521c0d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.380+0000 7f5219e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f521c15e060 0x7f521c0d0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40214/0 (socket says 172.21.15.82:40214) 2024-03-16T17:58:12.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.380+0000 7f5219e00640 1 -- 172.21.15.82:0/3297087377 learned_addr learned my addr 172.21.15.82:0/3297087377 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:12.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.380+0000 7f521a800640 1 --1- 172.21.15.82:0/3297087377 >> v1:172.21.15.82:6789/0 conn(0x7f521c0d27a0 0x7f521c0ce890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51266/0 (socket says 172.21.15.82:51266) 2024-03-16T17:58:12.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.381+0000 7f5211400640 1 --2- 172.21.15.82:0/3297087377 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f521c0b2bb0 0x7f521c0d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.381+0000 7f5219e00640 1 -- 172.21.15.82:0/3297087377 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f521c0b2bb0 msgr2=0x7f521c0d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:12.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.381+0000 7f5219e00640 1 --2- 172.21.15.82:0/3297087377 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f521c0b2bb0 0x7f521c0d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.381+0000 7f5219e00640 1 -- 172.21.15.82:0/3297087377 >> v1:172.21.15.82:6789/0 conn(0x7f521c0d27a0 legacy=0x7f521c0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:12.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.381+0000 7f5219e00640 1 -- 172.21.15.82:0/3297087377 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c0d4840 con 0x7f521c15e060 2024-03-16T17:58:12.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.381+0000 7f5219400640 1 -- 172.21.15.82:0/3297087377 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568027566 0 0) 0x7f521c0d1de0 con 0x7f521c0d27a0 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.382+0000 7f5219e00640 1 --2- 172.21.15.82:0/3297087377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c15e060 0x7f521c0d0ce0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f520c009870 tx=0x7f520c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7917a90d9c5acc81 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.382+0000 7f5219400640 1 -- 172.21.15.82:0/3297087377 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c030840 con 0x7f521c15e060 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.382+0000 7f5219400640 1 -- 172.21.15.82:0/3297087377 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c0309a0 con 0x7f521c15e060 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.382+0000 7f5219400640 1 -- 172.21.15.82:0/3297087377 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c030d30 con 0x7f521c15e060 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.383+0000 7f521b200640 1 -- 172.21.15.82:0/3297087377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c15e060 msgr2=0x7f521c0d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.383+0000 7f521b200640 1 --2- 172.21.15.82:0/3297087377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c15e060 0x7f521c0d0ce0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f520c009870 tx=0x7f520c02b980 comp rx=0 tx=0).stop 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.383+0000 7f521b200640 1 -- 172.21.15.82:0/3297087377 shutdown_connections 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.383+0000 7f521b200640 1 --2- 172.21.15.82:0/3297087377 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f521c0b2bb0 0x7f521c0d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.383+0000 7f521b200640 1 --2- 172.21.15.82:0/3297087377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c15e060 0x7f521c0d0ce0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.383+0000 7f521b200640 1 -- 172.21.15.82:0/3297087377 >> 172.21.15.82:0/3297087377 conn(0x7f521c0d3c80 msgr2=0x7f521c0d4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.383+0000 7f521b200640 1 -- 172.21.15.82:0/3297087377 shutdown_connections 2024-03-16T17:58:12.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.383+0000 7f521b200640 1 -- 172.21.15.82:0/3297087377 wait complete. 2024-03-16T17:58:12.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.385+0000 7f521b200640 1 Processor -- start 2024-03-16T17:58:12.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.385+0000 7f521b200640 1 -- start start 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.386+0000 7f521b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c0b2bb0 0x7f521c15c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.386+0000 7f521b200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f521c0d27a0 0x7f521c0b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.386+0000 7f521b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f521c15e060 0x7f521c0b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.386+0000 7f521b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0cade0 con 0x7f521c0b2bb0 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.386+0000 7f521b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c1f7850 con 0x7f521c15e060 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.386+0000 7f521b200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c1f7b30 con 0x7f521c0d27a0 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.386+0000 7f5219e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c0b2bb0 0x7f521c15c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.386+0000 7f5219e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c0b2bb0 0x7f521c15c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40224/0 (socket says 172.21.15.82:40224) 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.386+0000 7f5219e00640 1 -- 172.21.15.82:0/3203194004 learned_addr learned my addr 172.21.15.82:0/3203194004 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f521a800640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f521c15e060 0x7f521c0b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f5211400640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f521c0d27a0 0x7f521c0b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f5219e00640 1 -- 172.21.15.82:0/3203194004 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f521c0d27a0 msgr2=0x7f521c0b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f5219e00640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f521c0d27a0 0x7f521c0b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f5219e00640 1 -- 172.21.15.82:0/3203194004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f521c15e060 msgr2=0x7f521c0b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f5219e00640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f521c15e060 0x7f521c0b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f5219e00640 1 -- 172.21.15.82:0/3203194004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520c0094d0 con 0x7f521c0b2bb0 2024-03-16T17:58:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f5219e00640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c0b2bb0 0x7f521c15c3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f520c000c00 tx=0x7f520c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f5212a00640 1 -- 172.21.15.82:0/3203194004 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c03e070 con 0x7f521c0b2bb0 2024-03-16T17:58:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f5212a00640 1 -- 172.21.15.82:0/3203194004 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520c004030 con 0x7f521c0b2bb0 2024-03-16T17:58:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f5212a00640 1 -- 172.21.15.82:0/3203194004 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520c0043c0 con 0x7f521c0b2bb0 2024-03-16T17:58:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f521b200640 1 -- 172.21.15.82:0/3203194004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c1f7e10 con 0x7f521c0b2bb0 2024-03-16T17:58:12.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.387+0000 7f521b200640 1 -- 172.21.15.82:0/3203194004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c0a1470 con 0x7f521c0b2bb0 2024-03-16T17:58:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.389+0000 7f521b200640 1 -- 172.21.15.82:0/3203194004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c04f250 con 0x7f521c0b2bb0 2024-03-16T17:58:12.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.389+0000 7f5212a00640 1 -- 172.21.15.82:0/3203194004 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f520c042070 con 0x7f521c0b2bb0 2024-03-16T17:58:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.390+0000 7f5212a00640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f51f806b3b0 0x7f51f806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.390+0000 7f5212a00640 1 -- 172.21.15.82:0/3203194004 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f520c0abb30 con 0x7f521c0b2bb0 2024-03-16T17:58:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.391+0000 7f5211400640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f51f806b3b0 0x7f51f806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.391+0000 7f5211400640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f51f806b3b0 0x7f51f806d870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f521c0b21b0 tx=0x7f5204008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:12.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.395+0000 7f5212a00640 1 -- 172.21.15.82:0/3203194004 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f520c076810 con 0x7f521c0b2bb0 2024-03-16T17:58:12.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.593+0000 7f521b200640 1 -- 172.21.15.82:0/3203194004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f521c0c7550 con 0x7f521c0b2bb0 2024-03-16T17:58:12.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.596+0000 7f5212a00640 1 -- 172.21.15.82:0/3203194004 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f520c034020 con 0x7f521c0b2bb0 2024-03-16T17:58:12.600 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:12.600 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:12.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 -- 172.21.15.82:0/3203194004 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f51f806b3b0 msgr2=0x7f51f806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:12.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f51f806b3b0 0x7f51f806d870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f521c0b21b0 tx=0x7f5204008040 comp rx=0 tx=0).stop 2024-03-16T17:58:12.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 -- 172.21.15.82:0/3203194004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c0b2bb0 msgr2=0x7f521c15c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:12.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c0b2bb0 0x7f521c15c3c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f520c000c00 tx=0x7f520c0046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 -- 172.21.15.82:0/3203194004 shutdown_connections 2024-03-16T17:58:12.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f51f806b3b0 0x7f51f806d870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f521c15e060 0x7f521c0b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f521c0d27a0 0x7f521c0b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 --2- 172.21.15.82:0/3203194004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f521c0b2bb0 0x7f521c15c3c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 -- 172.21.15.82:0/3203194004 >> 172.21.15.82:0/3203194004 conn(0x7f521c0d3c80 msgr2=0x7f521c0c77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:12.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 -- 172.21.15.82:0/3203194004 shutdown_connections 2024-03-16T17:58:12.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.599+0000 7f521b200640 1 -- 172.21.15.82:0/3203194004 wait complete. 2024-03-16T17:58:12.622 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:12.623 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.826+0000 7f8cbf200640 1 Processor -- start 2024-03-16T17:58:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.826+0000 7f8cbf200640 1 -- start start 2024-03-16T17:58:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.827+0000 7f8cbf200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8cb80cca20 0x7f8cb8166be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.827+0000 7f8cbf200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb80ce550 con 0x7f8cb8159960 2024-03-16T17:58:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.827+0000 7f8cbf200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb815c370 con 0x7f8cb806e940 2024-03-16T17:58:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.827+0000 7f8cbf200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb8159d40 con 0x7f8cb80cca20 2024-03-16T17:58:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.828+0000 7f8cbd200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8cb80cca20 0x7f8cb8166be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.828+0000 7f8cbd200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8cb80cca20 0x7f8cb8166be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40230/0 (socket says 172.21.15.82:40230) 2024-03-16T17:58:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.828+0000 7f8cbd200640 1 -- 172.21.15.82:0/3097234353 learned_addr learned my addr 172.21.15.82:0/3097234353 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.828+0000 7f8cbd200640 1 -- 172.21.15.82:0/3097234353 >> v1:172.21.15.90:6789/0 conn(0x7f8cb806e940 legacy=0x7f8cb80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.828+0000 7f8cbd200640 1 -- 172.21.15.82:0/3097234353 >> v1:172.21.15.82:6789/0 conn(0x7f8cb8159960 legacy=0x7f8cb806e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.828+0000 7f8cbd200640 1 -- 172.21.15.82:0/3097234353 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb815c650 con 0x7f8cb80cca20 2024-03-16T17:58:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.829+0000 7f8cbd200640 1 --2- 172.21.15.82:0/3097234353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb80cca20 0x7f8cb8166be0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009870 tx=0x7f8cb002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955432ff4071e113 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.829+0000 7f8cb7400640 1 -- 172.21.15.82:0/3097234353 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0031870 con 0x7f8cb80cca20 2024-03-16T17:58:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.829+0000 7f8cb7400640 1 -- 172.21.15.82:0/3097234353 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb00319d0 con 0x7f8cb80cca20 2024-03-16T17:58:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.829+0000 7f8cb7400640 1 -- 172.21.15.82:0/3097234353 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0031d60 con 0x7f8cb80cca20 2024-03-16T17:58:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.830+0000 7f8cbf200640 1 -- 172.21.15.82:0/3097234353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb80cca20 msgr2=0x7f8cb8166be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.830+0000 7f8cbf200640 1 --2- 172.21.15.82:0/3097234353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb80cca20 0x7f8cb8166be0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009870 tx=0x7f8cb002b910 comp rx=0 tx=0).stop 2024-03-16T17:58:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.830+0000 7f8cbf200640 1 -- 172.21.15.82:0/3097234353 shutdown_connections 2024-03-16T17:58:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.830+0000 7f8cbf200640 1 --2- 172.21.15.82:0/3097234353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb80cca20 0x7f8cb8166be0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.830+0000 7f8cbf200640 1 -- 172.21.15.82:0/3097234353 >> 172.21.15.82:0/3097234353 conn(0x7f8cb80b1640 msgr2=0x7f8cb80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.830+0000 7f8cbf200640 1 -- 172.21.15.82:0/3097234353 shutdown_connections 2024-03-16T17:58:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.830+0000 7f8cbf200640 1 -- 172.21.15.82:0/3097234353 wait complete. 2024-03-16T17:58:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.832+0000 7f8cbf200640 1 Processor -- start 2024-03-16T17:58:12.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.832+0000 7f8cbf200640 1 -- start start 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbf200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8cb806e940 0x7f8cb80b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbf200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cb80cca20 0x7f8cb81437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbf200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb8159960 0x7f8cb8143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbf200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb815cff0 con 0x7f8cb8159960 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbf200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb8142400 con 0x7f8cb80cca20 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbf200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb81443b0 con 0x7f8cb806e940 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbd200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb8159960 0x7f8cb8143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbd200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb8159960 0x7f8cb8143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40244/0 (socket says 172.21.15.82:40244) 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbd200640 1 -- 172.21.15.82:0/1360679504 learned_addr learned my addr 172.21.15.82:0/1360679504 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cb7e00640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8cb806e940 0x7f8cb80b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbd200640 1 -- 172.21.15.82:0/1360679504 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8cb806e940 msgr2=0x7f8cb80b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:12.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbd200640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8cb806e940 0x7f8cb80b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8caf400640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cb80cca20 0x7f8cb81437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbd200640 1 -- 172.21.15.82:0/1360679504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cb80cca20 msgr2=0x7f8cb81437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:12.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbd200640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cb80cca20 0x7f8cb81437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:12.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.833+0000 7f8cbd200640 1 -- 172.21.15.82:0/1360679504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb00094d0 con 0x7f8cb8159960 2024-03-16T17:58:12.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.834+0000 7f8cbd200640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb8159960 0x7f8cb8143d30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8cb002b950 tx=0x7f8cb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:12.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.834+0000 7f8cb4c00640 1 -- 172.21.15.82:0/1360679504 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00443f0 con 0x7f8cb8159960 2024-03-16T17:58:12.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.834+0000 7f8cb4c00640 1 -- 172.21.15.82:0/1360679504 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0044570 con 0x7f8cb8159960 2024-03-16T17:58:12.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.834+0000 7f8cb4c00640 1 -- 172.21.15.82:0/1360679504 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0044920 con 0x7f8cb8159960 2024-03-16T17:58:12.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.834+0000 7f8cbf200640 1 -- 172.21.15.82:0/1360679504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb8144690 con 0x7f8cb8159960 2024-03-16T17:58:12.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.834+0000 7f8cbf200640 1 -- 172.21.15.82:0/1360679504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb8144c50 con 0x7f8cb8159960 2024-03-16T17:58:12.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.835+0000 7f8cbf200640 1 -- 172.21.15.82:0/1360679504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb8067f30 con 0x7f8cb8159960 2024-03-16T17:58:12.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.836+0000 7f8cb4c00640 1 -- 172.21.15.82:0/1360679504 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8cb004d450 con 0x7f8cb8159960 2024-03-16T17:58:12.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.837+0000 7f8cb4c00640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8c9406b3b0 0x7f8c9406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.837+0000 7f8cb4c00640 1 -- 172.21.15.82:0/1360679504 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8cb002d040 con 0x7f8cb8159960 2024-03-16T17:58:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.837+0000 7f8cb7e00640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8c9406b3b0 0x7f8c9406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.838+0000 7f8cb7e00640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8c9406b3b0 0x7f8c9406d870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8007fd0 tx=0x7f8ca8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:12.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:12.842+0000 7f8cb4c00640 1 -- 172.21.15.82:0/1360679504 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8cb0077050 con 0x7f8cb8159960 2024-03-16T17:58:13.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.036+0000 7f8cbf200640 1 -- 172.21.15.82:0/1360679504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8cb80ceb80 con 0x7f8cb8159960 2024-03-16T17:58:13.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.037+0000 7f8cb4c00640 1 -- 172.21.15.82:0/1360679504 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8cb007a710 con 0x7f8cb8159960 2024-03-16T17:58:13.041 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 -- 172.21.15.82:0/1360679504 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8c9406b3b0 msgr2=0x7f8c9406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8c9406b3b0 0x7f8c9406d870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8007fd0 tx=0x7f8ca8009040 comp rx=0 tx=0).stop 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 -- 172.21.15.82:0/1360679504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb8159960 msgr2=0x7f8cb8143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb8159960 0x7f8cb8143d30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8cb002b950 tx=0x7f8cb00046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 -- 172.21.15.82:0/1360679504 shutdown_connections 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8c9406b3b0 0x7f8c9406d870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8cb8159960 0x7f8cb8143d30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cb80cca20 0x7f8cb81437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 --2- 172.21.15.82:0/1360679504 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8cb806e940 0x7f8cb80b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 -- 172.21.15.82:0/1360679504 >> 172.21.15.82:0/1360679504 conn(0x7f8cb80b1640 msgr2=0x7f8cb80af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 -- 172.21.15.82:0/1360679504 shutdown_connections 2024-03-16T17:58:13.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.040+0000 7f8cbf200640 1 -- 172.21.15.82:0/1360679504 wait complete. 2024-03-16T17:58:13.062 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:58:13.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.267+0000 7fd04ea00640 1 Processor -- start 2024-03-16T17:58:13.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.267+0000 7fd04ea00640 1 -- start start 2024-03-16T17:58:13.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.269+0000 7fd04ea00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd048144330 0x7fd048070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.269+0000 7fd04ea00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480cc380 0x7fd0480cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.269+0000 7fd04ea00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0480ce8f0 con 0x7fd0480714f0 2024-03-16T17:58:13.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.269+0000 7fd04ea00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0480ccd50 con 0x7fd048144330 2024-03-16T17:58:13.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.269+0000 7fd04ea00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04815f2d0 con 0x7fd0480cc380 2024-03-16T17:58:13.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.270+0000 7fd047e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd048144330 0x7fd048070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.270+0000 7fd04ca00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480cc380 0x7fd0480cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.270+0000 7fd04ca00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480cc380 0x7fd0480cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:35510/0 (socket says 172.21.15.82:35510) 2024-03-16T17:58:13.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.270+0000 7fd04ca00640 1 -- 172.21.15.82:0/3174517265 learned_addr learned my addr 172.21.15.82:0/3174517265 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:13.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.271+0000 7fd047e00640 1 -- 172.21.15.82:0/3174517265 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480cc380 msgr2=0x7fd0480cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:13.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.271+0000 7fd047e00640 1 --2- 172.21.15.82:0/3174517265 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480cc380 0x7fd0480cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.271+0000 7fd047e00640 1 -- 172.21.15.82:0/3174517265 >> v1:172.21.15.90:6789/0 conn(0x7fd0480714f0 legacy=0x7fd04815e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.271+0000 7fd047e00640 1 -- 172.21.15.82:0/3174517265 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04815f5b0 con 0x7fd048144330 2024-03-16T17:58:13.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.272+0000 7fd047e00640 1 --2- 172.21.15.82:0/3174517265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd048144330 0x7fd048070fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd0380097a0 tx=0x7fd03802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9077bebd093a395d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:13.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.272+0000 7fd047400640 1 -- 172.21.15.82:0/3174517265 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03802d040 con 0x7fd048144330 2024-03-16T17:58:13.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.272+0000 7fd047400640 1 -- 172.21.15.82:0/3174517265 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd038031810 con 0x7fd048144330 2024-03-16T17:58:13.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.272+0000 7fd047400640 1 -- 172.21.15.82:0/3174517265 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038031ba0 con 0x7fd048144330 2024-03-16T17:58:13.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.273+0000 7fd04ea00640 1 -- 172.21.15.82:0/3174517265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd048144330 msgr2=0x7fd048070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.273+0000 7fd04ea00640 1 --2- 172.21.15.82:0/3174517265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd048144330 0x7fd048070fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd0380097a0 tx=0x7fd03802b8b0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.273+0000 7fd04ea00640 1 -- 172.21.15.82:0/3174517265 shutdown_connections 2024-03-16T17:58:13.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.273+0000 7fd04ea00640 1 --2- 172.21.15.82:0/3174517265 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480cc380 0x7fd0480cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.273+0000 7fd04ea00640 1 --2- 172.21.15.82:0/3174517265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd048144330 0x7fd048070fb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.273+0000 7fd04ea00640 1 -- 172.21.15.82:0/3174517265 >> 172.21.15.82:0/3174517265 conn(0x7fd0480b1640 msgr2=0x7fd0480b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:13.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.273+0000 7fd04ea00640 1 -- 172.21.15.82:0/3174517265 shutdown_connections 2024-03-16T17:58:13.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.273+0000 7fd04ea00640 1 -- 172.21.15.82:0/3174517265 wait complete. 2024-03-16T17:58:13.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.275+0000 7fd04ea00640 1 Processor -- start 2024-03-16T17:58:13.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.275+0000 7fd04ea00640 1 -- start start 2024-03-16T17:58:13.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ea00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480714f0 0x7fd0481f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ea00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0480cc380 0x7fd0480b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd048144330 0x7fd0480b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ea00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04815e0c0 con 0x7fd048144330 2024-03-16T17:58:13.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ea00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480b2d10 con 0x7fd0480cc380 2024-03-16T17:58:13.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ea00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0480b56d0 con 0x7fd0480714f0 2024-03-16T17:58:13.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd048144330 0x7fd0480b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd048144330 0x7fd0480b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40254/0 (socket says 172.21.15.82:40254) 2024-03-16T17:58:13.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ca00640 1 -- 172.21.15.82:0/1169913494 learned_addr learned my addr 172.21.15.82:0/1169913494 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:13.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd03f400640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0480cc380 0x7fd0480b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd047e00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480714f0 0x7fd0481f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ca00640 1 -- 172.21.15.82:0/1169913494 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480714f0 msgr2=0x7fd0481f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ca00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480714f0 0x7fd0481f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ca00640 1 -- 172.21.15.82:0/1169913494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0480cc380 msgr2=0x7fd0480b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ca00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0480cc380 0x7fd0480b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.276+0000 7fd04ca00640 1 -- 172.21.15.82:0/1169913494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd038009400 con 0x7fd048144330 2024-03-16T17:58:13.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.277+0000 7fd04ca00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd048144330 0x7fd0480b3ef0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd040009d90 tx=0x7fd04000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:13.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.277+0000 7fd044c00640 1 -- 172.21.15.82:0/1169913494 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04000cd20 con 0x7fd048144330 2024-03-16T17:58:13.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.277+0000 7fd044c00640 1 -- 172.21.15.82:0/1169913494 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04000f040 con 0x7fd048144330 2024-03-16T17:58:13.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.277+0000 7fd044c00640 1 -- 172.21.15.82:0/1169913494 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040003bb0 con 0x7fd048144330 2024-03-16T17:58:13.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.277+0000 7fd04ea00640 1 -- 172.21.15.82:0/1169913494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04815c870 con 0x7fd048144330 2024-03-16T17:58:13.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.277+0000 7fd04ea00640 1 -- 172.21.15.82:0/1169913494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04815ce30 con 0x7fd048144330 2024-03-16T17:58:13.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.278+0000 7fd04ea00640 1 -- 172.21.15.82:0/1169913494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0480b2ec0 con 0x7fd048144330 2024-03-16T17:58:13.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.279+0000 7fd044c00640 1 -- 172.21.15.82:0/1169913494 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fd040003d10 con 0x7fd048144330 2024-03-16T17:58:13.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.280+0000 7fd044c00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd02406b3b0 0x7fd02406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.280+0000 7fd044c00640 1 -- 172.21.15.82:0/1169913494 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd04008a1e0 con 0x7fd048144330 2024-03-16T17:58:13.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.280+0000 7fd047e00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd02406b3b0 0x7fd02406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.281+0000 7fd047e00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd02406b3b0 0x7fd02406d870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd038036000 tx=0x7fd03803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:13.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.284+0000 7fd044c00640 1 -- 172.21.15.82:0/1169913494 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0480b2ec0 con 0x7fd048144330 2024-03-16T17:58:13.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.473+0000 7fd04ea00640 1 -- 172.21.15.82:0/1169913494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0480b2ec0 con 0x7fd048144330 2024-03-16T17:58:13.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.473+0000 7fd044c00640 1 -- 172.21.15.82:0/1169913494 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd0480b2ec0 con 0x7fd048144330 2024-03-16T17:58:13.477 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:58:13.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 -- 172.21.15.82:0/1169913494 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd02406b3b0 msgr2=0x7fd02406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd02406b3b0 0x7fd02406d870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd038036000 tx=0x7fd03803c040 comp rx=0 tx=0).stop 2024-03-16T17:58:13.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 -- 172.21.15.82:0/1169913494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd048144330 msgr2=0x7fd0480b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd048144330 0x7fd0480b3ef0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd040009d90 tx=0x7fd04000c490 comp rx=0 tx=0).stop 2024-03-16T17:58:13.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 -- 172.21.15.82:0/1169913494 shutdown_connections 2024-03-16T17:58:13.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd02406b3b0 0x7fd02406d870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd048144330 0x7fd0480b3ef0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0480cc380 0x7fd0480b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 --2- 172.21.15.82:0/1169913494 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd0480714f0 0x7fd0481f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 -- 172.21.15.82:0/1169913494 >> 172.21.15.82:0/1169913494 conn(0x7fd0480b1640 msgr2=0x7fd0480b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:13.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 -- 172.21.15.82:0/1169913494 shutdown_connections 2024-03-16T17:58:13.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.476+0000 7fd04ea00640 1 -- 172.21.15.82:0/1169913494 wait complete. 2024-03-16T17:58:13.497 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:58:13.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.700+0000 7fc8e1200640 1 Processor -- start 2024-03-16T17:58:13.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.700+0000 7fc8e1200640 1 -- start start 2024-03-16T17:58:13.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.702+0000 7fc8e1200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8dc0d0790 0x7fc8dc0d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.702+0000 7fc8e1200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8dc0d0b70 0x7fc8dc0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.702+0000 7fc8e1200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8dc0d4a40 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.702+0000 7fc8e1200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8dc0d4d20 con 0x7fc8dc0d0790 2024-03-16T17:58:13.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.702+0000 7fc8e1200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0b11b0 con 0x7fc8dc0d0b70 2024-03-16T17:58:13.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.702+0000 7fc8d9e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8dc0d0790 0x7fc8dc0d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.702+0000 7fc8d9e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8dc0d0790 0x7fc8dc0d1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40262/0 (socket says 172.21.15.82:40262) 2024-03-16T17:58:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.702+0000 7fc8d9e00640 1 -- 172.21.15.82:0/1201057190 learned_addr learned my addr 172.21.15.82:0/1201057190 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.702+0000 7fc8da800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc8dc0b1b10 0x7fc8dc0b1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51280/0 (socket says 172.21.15.82:51280) 2024-03-16T17:58:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.702+0000 7fc8d1400640 1 --2- 172.21.15.82:0/1201057190 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8dc0d0b70 0x7fc8dc0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.703+0000 7fc8d9e00640 1 -- 172.21.15.82:0/1201057190 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8dc0d0b70 msgr2=0x7fc8dc0b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.703+0000 7fc8d9e00640 1 --2- 172.21.15.82:0/1201057190 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8dc0d0b70 0x7fc8dc0b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.703+0000 7fc8d9e00640 1 -- 172.21.15.82:0/1201057190 >> v1:172.21.15.82:6789/0 conn(0x7fc8dc0b1b10 legacy=0x7fc8dc0b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.703+0000 7fc8d9e00640 1 -- 172.21.15.82:0/1201057190 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc0b0af0 con 0x7fc8dc0d0790 2024-03-16T17:58:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.703+0000 7fc8d9400640 1 -- 172.21.15.82:0/1201057190 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969495462 0 0) 0x7fc8dc0d4a40 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.704+0000 7fc8d9e00640 1 --2- 172.21.15.82:0/1201057190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0d0790 0x7fc8dc0d1b60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009e50 tx=0x7fc8cc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327c2c43a28c0d18 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:13.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.704+0000 7fc8d9400640 1 -- 172.21.15.82:0/1201057190 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc02cb00 con 0x7fc8dc0d0790 2024-03-16T17:58:13.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.704+0000 7fc8d9400640 1 -- 172.21.15.82:0/1201057190 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc02cc60 con 0x7fc8dc0d0790 2024-03-16T17:58:13.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.704+0000 7fc8d9400640 1 -- 172.21.15.82:0/1201057190 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0343f0 con 0x7fc8dc0d0790 2024-03-16T17:58:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.705+0000 7fc8e1200640 1 -- 172.21.15.82:0/1201057190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0d0790 msgr2=0x7fc8dc0d1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.705+0000 7fc8e1200640 1 --2- 172.21.15.82:0/1201057190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0d0790 0x7fc8dc0d1b60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009e50 tx=0x7fc8cc02bf60 comp rx=0 tx=0).stop 2024-03-16T17:58:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.705+0000 7fc8e1200640 1 -- 172.21.15.82:0/1201057190 shutdown_connections 2024-03-16T17:58:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.705+0000 7fc8e1200640 1 --2- 172.21.15.82:0/1201057190 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8dc0d0b70 0x7fc8dc0b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.705+0000 7fc8e1200640 1 --2- 172.21.15.82:0/1201057190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0d0790 0x7fc8dc0d1b60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.705+0000 7fc8e1200640 1 -- 172.21.15.82:0/1201057190 >> 172.21.15.82:0/1201057190 conn(0x7fc8dc0d3d80 msgr2=0x7fc8dc0d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.705+0000 7fc8e1200640 1 -- 172.21.15.82:0/1201057190 shutdown_connections 2024-03-16T17:58:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.705+0000 7fc8e1200640 1 -- 172.21.15.82:0/1201057190 wait complete. 2024-03-16T17:58:13.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.707+0000 7fc8e1200640 1 Processor -- start 2024-03-16T17:58:13.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.707+0000 7fc8e1200640 1 -- start start 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8e1200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0b1b10 0x7fc8dc0ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8e1200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8dc0d0790 0x7fc8dc154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8e1200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8dc0d0b70 0x7fc8dc0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8e1200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0cdc30 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8e1200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0cb210 con 0x7fc8dc0d0b70 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8e1200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0caca0 con 0x7fc8dc0d0790 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8d9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0b1b10 0x7fc8dc0ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8d9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0b1b10 0x7fc8dc0ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40270/0 (socket says 172.21.15.82:40270) 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8d9e00640 1 -- 172.21.15.82:0/481387113 learned_addr learned my addr 172.21.15.82:0/481387113 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8d1400640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8dc0d0790 0x7fc8dc154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8d9e00640 1 -- 172.21.15.82:0/481387113 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8dc0d0790 msgr2=0x7fc8dc154270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8d9e00640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8dc0d0790 0x7fc8dc154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8d9e00640 1 -- 172.21.15.82:0/481387113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8dc0d0b70 msgr2=0x7fc8dc0cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8d9e00640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8dc0d0b70 0x7fc8dc0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.708+0000 7fc8d9e00640 1 -- 172.21.15.82:0/481387113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc009b30 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.709+0000 7fc8d9e00640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0b1b10 0x7fc8dc0ca760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc036040 tx=0x7fc8cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:13.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.709+0000 7fc8d2a00640 1 -- 172.21.15.82:0/481387113 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc009510 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.709+0000 7fc8d2a00640 1 -- 172.21.15.82:0/481387113 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc03e450 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.709+0000 7fc8e1200640 1 -- 172.21.15.82:0/481387113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc1547b0 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.709+0000 7fc8d2a00640 1 -- 172.21.15.82:0/481387113 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc03e7c0 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.709+0000 7fc8e1200640 1 -- 172.21.15.82:0/481387113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8dc0cd290 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.710+0000 7fc8e1200640 1 -- 172.21.15.82:0/481387113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8dc067f30 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.711+0000 7fc8d2a00640 1 -- 172.21.15.82:0/481387113 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fc8cc03e920 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.712+0000 7fc8d2a00640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8b806b3b0 0x7fc8b806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:13.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.712+0000 7fc8d2a00640 1 -- 172.21.15.82:0/481387113 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8cc0acb40 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.712+0000 7fc8d1400640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8b806b3b0 0x7fc8b806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:13.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.713+0000 7fc8d1400640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8b806b3b0 0x7fc8b806d870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0b9760 tx=0x7fc8c4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:13.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.716+0000 7fc8d2a00640 1 -- 172.21.15.82:0/481387113 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc8cc077670 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.897+0000 7fc8e1200640 1 -- 172.21.15.82:0/481387113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8dc0d2140 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.898+0000 7fc8d2a00640 1 -- 172.21.15.82:0/481387113 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8cc0430a0 con 0x7fc8dc0b1b10 2024-03-16T17:58:13.902 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:58:13.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 -- 172.21.15.82:0/481387113 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8b806b3b0 msgr2=0x7fc8b806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8b806b3b0 0x7fc8b806d870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0b9760 tx=0x7fc8c4008040 comp rx=0 tx=0).stop 2024-03-16T17:58:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 -- 172.21.15.82:0/481387113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0b1b10 msgr2=0x7fc8dc0ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0b1b10 0x7fc8dc0ca760 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc036040 tx=0x7fc8cc004a40 comp rx=0 tx=0).stop 2024-03-16T17:58:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 -- 172.21.15.82:0/481387113 shutdown_connections 2024-03-16T17:58:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8b806b3b0 0x7fc8b806d870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8dc0d0b70 0x7fc8dc0cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8dc0d0790 0x7fc8dc154270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 --2- 172.21.15.82:0/481387113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8dc0b1b10 0x7fc8dc0ca760 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 -- 172.21.15.82:0/481387113 >> 172.21.15.82:0/481387113 conn(0x7fc8dc0d3d80 msgr2=0x7fc8dc160670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 -- 172.21.15.82:0/481387113 shutdown_connections 2024-03-16T17:58:13.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:13.901+0000 7fc8e1200640 1 -- 172.21.15.82:0/481387113 wait complete. 2024-03-16T17:58:13.923 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:14.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.126+0000 7f7fc3400640 1 Processor -- start 2024-03-16T17:58:14.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.127+0000 7f7fc3400640 1 -- start start 2024-03-16T17:58:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.128+0000 7f7fc3400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7fbc15e210 0x7f7fbc0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.128+0000 7f7fc3400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fbc06c190 0x7f7fbc06c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.128+0000 7f7fc3400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7fbc06ced0 0x7f7fbc161c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.128+0000 7f7fc3400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fbc0d0d20 con 0x7f7fbc06ced0 2024-03-16T17:58:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.128+0000 7f7fc3400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fbc15e5f0 con 0x7f7fbc15e210 2024-03-16T17:58:14.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.128+0000 7f7fc3400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc06d340 con 0x7f7fbc06c190 2024-03-16T17:58:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.128+0000 7f7fc1400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7fbc06ced0 0x7f7fbc161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.129+0000 7f7fc1400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7fbc06ced0 0x7f7fbc161c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40274/0 (socket says 172.21.15.82:40274) 2024-03-16T17:58:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.129+0000 7f7fc1400640 1 -- 172.21.15.82:0/701461824 learned_addr learned my addr 172.21.15.82:0/701461824 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.129+0000 7f7fbbe00640 1 --2- 172.21.15.82:0/701461824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fbc06c190 0x7f7fbc06c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.129+0000 7f7fc0a00640 1 --2- 172.21.15.82:0/701461824 >> v2:172.21.15.90:3300/0 conn(0x7f7fbc15e210 0x7f7fbc0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.129+0000 7f7fc1400640 1 -- 172.21.15.82:0/701461824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fbc06c190 msgr2=0x7f7fbc06c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.129+0000 7f7fc1400640 1 --2- 172.21.15.82:0/701461824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fbc06c190 0x7f7fbc06c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.129+0000 7f7fc1400640 1 -- 172.21.15.82:0/701461824 >> v2:172.21.15.90:3300/0 conn(0x7f7fbc15e210 msgr2=0x7f7fbc0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.129+0000 7f7fc1400640 1 --2- 172.21.15.82:0/701461824 >> v2:172.21.15.90:3300/0 conn(0x7f7fbc15e210 0x7f7fbc0d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.129+0000 7f7fc1400640 1 -- 172.21.15.82:0/701461824 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc06ec40 con 0x7f7fbc06ced0 2024-03-16T17:58:14.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.130+0000 7f7fc1400640 1 --2- 172.21.15.82:0/701461824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 0x7f7fbc161c70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc070e90 tx=0x7f7fb402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159dd19f01e34cc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.130+0000 7f7fbb400640 1 -- 172.21.15.82:0/701461824 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4031640 con 0x7f7fbc06ced0 2024-03-16T17:58:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fbb400640 1 -- 172.21.15.82:0/701461824 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb40317a0 con 0x7f7fbc06ced0 2024-03-16T17:58:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fbb400640 1 -- 172.21.15.82:0/701461824 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4031b30 con 0x7f7fbc06ced0 2024-03-16T17:58:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fc3400640 1 -- 172.21.15.82:0/701461824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 msgr2=0x7f7fbc161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fc3400640 1 --2- 172.21.15.82:0/701461824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 0x7f7fbc161c70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc070e90 tx=0x7f7fb402b8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fc3400640 1 -- 172.21.15.82:0/701461824 shutdown_connections 2024-03-16T17:58:14.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fc3400640 1 --2- 172.21.15.82:0/701461824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 0x7f7fbc161c70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fc3400640 1 --2- 172.21.15.82:0/701461824 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fbc06c190 0x7f7fbc06c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fc3400640 1 --2- 172.21.15.82:0/701461824 >> v2:172.21.15.90:3300/0 conn(0x7f7fbc15e210 0x7f7fbc0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fc3400640 1 -- 172.21.15.82:0/701461824 >> 172.21.15.82:0/701461824 conn(0x7f7fbc0b3780 msgr2=0x7f7fbc0b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:14.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fc3400640 1 -- 172.21.15.82:0/701461824 shutdown_connections 2024-03-16T17:58:14.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.131+0000 7f7fc3400640 1 -- 172.21.15.82:0/701461824 wait complete. 2024-03-16T17:58:14.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.133+0000 7f7fc3400640 1 Processor -- start 2024-03-16T17:58:14.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.133+0000 7f7fc3400640 1 -- start start 2024-03-16T17:58:14.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.134+0000 7f7fc3400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fbc06c190 0x7f7fbc1f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.134+0000 7f7fc3400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 0x7f7fbc09f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.134+0000 7f7fc3400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fbc15e210 0x7f7fbc0c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.134+0000 7f7fc3400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc09fc10 con 0x7f7fbc06ced0 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.134+0000 7f7fc3400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc0c8020 con 0x7f7fbc15e210 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.134+0000 7f7fc3400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc0c6f80 con 0x7f7fbc06c190 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fbbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 0x7f7fbc09f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fbbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 0x7f7fbc09f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40280/0 (socket says 172.21.15.82:40280) 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fbbe00640 1 -- 172.21.15.82:0/42573969 learned_addr learned my addr 172.21.15.82:0/42573969 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fc0a00640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fbc06c190 0x7f7fbc1f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fc1400640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fbc15e210 0x7f7fbc0c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fbbe00640 1 -- 172.21.15.82:0/42573969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fbc06c190 msgr2=0x7f7fbc1f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fbbe00640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fbc06c190 0x7f7fbc1f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fbbe00640 1 -- 172.21.15.82:0/42573969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fbc15e210 msgr2=0x7f7fbc0c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fbbe00640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fbc15e210 0x7f7fbc0c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fbbe00640 1 -- 172.21.15.82:0/42573969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40094d0 con 0x7f7fbc06ced0 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fc1400640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fbc15e210 0x7f7fbc0c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.135+0000 7f7fbbe00640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 0x7f7fbc09f6d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800d800 tx=0x7f7fa800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:14.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.136+0000 7f7fb8c00640 1 -- 172.21.15.82:0/42573969 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa800b6c0 con 0x7f7fbc06ced0 2024-03-16T17:58:14.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.136+0000 7f7fb8c00640 1 -- 172.21.15.82:0/42573969 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa800b820 con 0x7f7fbc06ced0 2024-03-16T17:58:14.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.136+0000 7f7fb8c00640 1 -- 172.21.15.82:0/42573969 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa800bbb0 con 0x7f7fbc06ced0 2024-03-16T17:58:14.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.136+0000 7f7fc3400640 1 -- 172.21.15.82:0/42573969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc0c7260 con 0x7f7fbc06ced0 2024-03-16T17:58:14.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.136+0000 7f7fc3400640 1 -- 172.21.15.82:0/42573969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fbc0c7820 con 0x7f7fbc06ced0 2024-03-16T17:58:14.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.137+0000 7f7fc3400640 1 -- 172.21.15.82:0/42573969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fbc1614e0 con 0x7f7fbc06ced0 2024-03-16T17:58:14.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.138+0000 7f7fb8c00640 1 -- 172.21.15.82:0/42573969 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f7fa800bd10 con 0x7f7fbc06ced0 2024-03-16T17:58:14.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.139+0000 7f7fb8c00640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7f9c06b3b0 0x7f7f9c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.139+0000 7f7fb8c00640 1 -- 172.21.15.82:0/42573969 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fa808a240 con 0x7f7fbc06ced0 2024-03-16T17:58:14.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.139+0000 7f7fc0a00640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7f9c06b3b0 0x7f7f9c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.140+0000 7f7fc0a00640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7f9c06b3b0 0x7f7f9c06d870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009940 tx=0x7f7fb0003cd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:14.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.143+0000 7f7fb8c00640 1 -- 172.21.15.82:0/42573969 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7fa8054e00 con 0x7f7fbc06ced0 2024-03-16T17:58:14.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.340+0000 7f7fc3400640 1 -- 172.21.15.82:0/42573969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fbc067f30 con 0x7f7fbc06ced0 2024-03-16T17:58:14.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.343+0000 7f7fb8c00640 1 -- 172.21.15.82:0/42573969 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f7fa80584c0 con 0x7f7fbc06ced0 2024-03-16T17:58:14.347 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:14.347 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:14.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 -- 172.21.15.82:0/42573969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7f9c06b3b0 msgr2=0x7f7f9c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:14.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7f9c06b3b0 0x7f7f9c06d870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009940 tx=0x7f7fb0003cd0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 -- 172.21.15.82:0/42573969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 msgr2=0x7f7fbc09f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 0x7f7fbc09f6d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7fa800d800 tx=0x7f7fa800dcc0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 -- 172.21.15.82:0/42573969 shutdown_connections 2024-03-16T17:58:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7f9c06b3b0 0x7f7f9c06d870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7fbc15e210 0x7f7fbc0c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7fbc06ced0 0x7f7fbc09f6d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 --2- 172.21.15.82:0/42573969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7fbc06c190 0x7f7fbc1f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 -- 172.21.15.82:0/42573969 >> 172.21.15.82:0/42573969 conn(0x7f7fbc0b3780 msgr2=0x7f7fbc06ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 -- 172.21.15.82:0/42573969 shutdown_connections 2024-03-16T17:58:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.346+0000 7f7fc3400640 1 -- 172.21.15.82:0/42573969 wait complete. 2024-03-16T17:58:14.370 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:14.370 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:14.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.580+0000 7f4b4a800640 1 Processor -- start 2024-03-16T17:58:14.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.581+0000 7f4b4a800640 1 -- start start 2024-03-16T17:58:14.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.582+0000 7f4b4a800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4b4415f300 0x7f4b440aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.582+0000 7f4b4a800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b440aefa0 0x7f4b440cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.582+0000 7f4b4a800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b440af440 con 0x7f4b4415f6e0 2024-03-16T17:58:14.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.582+0000 7f4b4a800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b440af720 con 0x7f4b4415f300 2024-03-16T17:58:14.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.582+0000 7f4b4a800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b440afa00 con 0x7f4b440aefa0 2024-03-16T17:58:14.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.583+0000 7f4b43400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4b4415f300 0x7f4b440aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.583+0000 7f4b43400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4b4415f300 0x7f4b440aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40282/0 (socket says 172.21.15.82:40282) 2024-03-16T17:58:14.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.583+0000 7f4b43400640 1 -- 172.21.15.82:0/2833502900 learned_addr learned my addr 172.21.15.82:0/2833502900 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:14.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.583+0000 7f4b42a00640 1 --2- 172.21.15.82:0/2833502900 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b440aefa0 0x7f4b440cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.583+0000 7f4b43400640 1 -- 172.21.15.82:0/2833502900 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b440aefa0 msgr2=0x7f4b440cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:14.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.583+0000 7f4b43400640 1 --2- 172.21.15.82:0/2833502900 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b440aefa0 0x7f4b440cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.583+0000 7f4b43400640 1 -- 172.21.15.82:0/2833502900 >> v1:172.21.15.90:6789/0 conn(0x7f4b4415f6e0 legacy=0x7f4b4406ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:14.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.583+0000 7f4b43400640 1 -- 172.21.15.82:0/2833502900 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b440afce0 con 0x7f4b4415f300 2024-03-16T17:58:14.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.584+0000 7f4b43400640 1 --2- 172.21.15.82:0/2833502900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b4415f300 0x7f4b440aea60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009870 tx=0x7f4b2c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c6cd2f6476f94c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:14.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.585+0000 7f4b42000640 1 -- 172.21.15.82:0/2833502900 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c031640 con 0x7f4b4415f300 2024-03-16T17:58:14.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.585+0000 7f4b42000640 1 -- 172.21.15.82:0/2833502900 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c0317a0 con 0x7f4b4415f300 2024-03-16T17:58:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.585+0000 7f4b42000640 1 -- 172.21.15.82:0/2833502900 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c031b30 con 0x7f4b4415f300 2024-03-16T17:58:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.585+0000 7f4b4a800640 1 -- 172.21.15.82:0/2833502900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b4415f300 msgr2=0x7f4b440aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.585+0000 7f4b4a800640 1 --2- 172.21.15.82:0/2833502900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b4415f300 0x7f4b440aea60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009870 tx=0x7f4b2c02b8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.586+0000 7f4b4a800640 1 -- 172.21.15.82:0/2833502900 shutdown_connections 2024-03-16T17:58:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.586+0000 7f4b4a800640 1 --2- 172.21.15.82:0/2833502900 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b440aefa0 0x7f4b440cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.586+0000 7f4b4a800640 1 --2- 172.21.15.82:0/2833502900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b4415f300 0x7f4b440aea60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.586+0000 7f4b4a800640 1 -- 172.21.15.82:0/2833502900 >> 172.21.15.82:0/2833502900 conn(0x7f4b440b37f0 msgr2=0x7f4b440b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.586+0000 7f4b4a800640 1 -- 172.21.15.82:0/2833502900 shutdown_connections 2024-03-16T17:58:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.586+0000 7f4b4a800640 1 -- 172.21.15.82:0/2833502900 wait complete. 2024-03-16T17:58:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.587+0000 7f4b4a800640 1 Processor -- start 2024-03-16T17:58:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.587+0000 7f4b4a800640 1 -- start start 2024-03-16T17:58:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.588+0000 7f4b4a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b440aefa0 0x7f4b44154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.588+0000 7f4b4a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4415f300 0x7f4b440b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.588+0000 7f4b4a800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4415f6e0 0x7f4b440b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.588+0000 7f4b4a800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b440b72c0 con 0x7f4b440aefa0 2024-03-16T17:58:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.588+0000 7f4b4a800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b440b66b0 con 0x7f4b4415f300 2024-03-16T17:58:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.588+0000 7f4b4a800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b440b6990 con 0x7f4b4415f6e0 2024-03-16T17:58:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b440aefa0 0x7f4b44154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b440aefa0 0x7f4b44154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40290/0 (socket says 172.21.15.82:40290) 2024-03-16T17:58:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43400640 1 -- 172.21.15.82:0/1396333472 learned_addr learned my addr 172.21.15.82:0/1396333472 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b42a00640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4415f300 0x7f4b440b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43400640 1 -- 172.21.15.82:0/1396333472 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4415f6e0 msgr2=0x7f4b440b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43e00640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4415f6e0 0x7f4b440b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43400640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4415f6e0 0x7f4b440b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43400640 1 -- 172.21.15.82:0/1396333472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4415f300 msgr2=0x7f4b440b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43400640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4415f300 0x7f4b440b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43400640 1 -- 172.21.15.82:0/1396333472 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b2c0094d0 con 0x7f4b440aefa0 2024-03-16T17:58:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43e00640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4415f6e0 0x7f4b440b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:14.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b42a00640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4415f300 0x7f4b440b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:14.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.589+0000 7f4b43400640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b440aefa0 0x7f4b44154180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009870 tx=0x7f4b2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:14.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.590+0000 7f4b3b400640 1 -- 172.21.15.82:0/1396333472 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c03ac60 con 0x7f4b440aefa0 2024-03-16T17:58:14.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.590+0000 7f4b3b400640 1 -- 172.21.15.82:0/1396333472 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c03adc0 con 0x7f4b440aefa0 2024-03-16T17:58:14.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.590+0000 7f4b4a800640 1 -- 172.21.15.82:0/1396333472 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b440b6c70 con 0x7f4b440aefa0 2024-03-16T17:58:14.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.590+0000 7f4b3b400640 1 -- 172.21.15.82:0/1396333472 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c044630 con 0x7f4b440aefa0 2024-03-16T17:58:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.590+0000 7f4b4a800640 1 -- 172.21.15.82:0/1396333472 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b440b6350 con 0x7f4b440aefa0 2024-03-16T17:58:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.591+0000 7f4b4a800640 1 -- 172.21.15.82:0/1396333472 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b440d1360 con 0x7f4b440aefa0 2024-03-16T17:58:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.592+0000 7f4b3b400640 1 -- 172.21.15.82:0/1396333472 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f4b2c02f050 con 0x7f4b440aefa0 2024-03-16T17:58:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.593+0000 7f4b3b400640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4b2006b3b0 0x7f4b2006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.593+0000 7f4b3b400640 1 -- 172.21.15.82:0/1396333472 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b2c0abbc0 con 0x7f4b440aefa0 2024-03-16T17:58:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.593+0000 7f4b42a00640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4b2006b3b0 0x7f4b2006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.594+0000 7f4b42a00640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4b2006b3b0 0x7f4b2006d870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4b440c8380 tx=0x7f4b3400b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.597+0000 7f4b3b400640 1 -- 172.21.15.82:0/1396333472 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4b2c076780 con 0x7f4b440aefa0 2024-03-16T17:58:14.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.785+0000 7f4b4a800640 1 -- 172.21.15.82:0/1396333472 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b44068640 con 0x7f4b440aefa0 2024-03-16T17:58:14.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.786+0000 7f4b3b400640 1 -- 172.21.15.82:0/1396333472 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b2c0350a0 con 0x7f4b440aefa0 2024-03-16T17:58:14.790 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:14.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.788+0000 7f4b4a800640 1 -- 172.21.15.82:0/1396333472 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4b2006b3b0 msgr2=0x7f4b2006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:14.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.788+0000 7f4b4a800640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4b2006b3b0 0x7f4b2006d870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4b440c8380 tx=0x7f4b3400b040 comp rx=0 tx=0).stop 2024-03-16T17:58:14.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.788+0000 7f4b4a800640 1 -- 172.21.15.82:0/1396333472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b440aefa0 msgr2=0x7f4b44154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:14.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.788+0000 7f4b4a800640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b440aefa0 0x7f4b44154180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009870 tx=0x7f4b2c0046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.788+0000 7f4b4a800640 1 -- 172.21.15.82:0/1396333472 shutdown_connections 2024-03-16T17:58:14.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.788+0000 7f4b4a800640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4b2006b3b0 0x7f4b2006d870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.788+0000 7f4b4a800640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4b4415f6e0 0x7f4b440b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.788+0000 7f4b4a800640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b4415f300 0x7f4b440b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.788+0000 7f4b4a800640 1 --2- 172.21.15.82:0/1396333472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b440aefa0 0x7f4b44154180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:14.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.788+0000 7f4b4a800640 1 -- 172.21.15.82:0/1396333472 >> 172.21.15.82:0/1396333472 conn(0x7f4b440b37f0 msgr2=0x7f4b440b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:14.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.789+0000 7f4b4a800640 1 -- 172.21.15.82:0/1396333472 shutdown_connections 2024-03-16T17:58:14.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:14.789+0000 7f4b4a800640 1 -- 172.21.15.82:0/1396333472 wait complete. 2024-03-16T17:58:14.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:58:15.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.021+0000 7fa8d7c00640 1 Processor -- start 2024-03-16T17:58:15.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.022+0000 7fa8d7c00640 1 -- start start 2024-03-16T17:58:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.023+0000 7fa8d7c00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa8d00d2ab0 0x7fa8d015e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.023+0000 7fa8d7c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d00d48e0 0x7fa8d00ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.023+0000 7fa8d7c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d00b33c0 con 0x7fa8d015e040 2024-03-16T17:58:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.023+0000 7fa8d7c00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d00d1cd0 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.023+0000 7fa8d7c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00d1fb0 con 0x7fa8d00d48e0 2024-03-16T17:58:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.024+0000 7fa8d5200640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa8d015e040 0x7fa8d00d23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47600/0 (socket says 172.21.15.82:47600) 2024-03-16T17:58:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.024+0000 7fa8d5200640 1 -- 172.21.15.82:0/283467393 learned_addr learned my addr 172.21.15.82:0/283467393 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.024+0000 7fa8d5c00640 1 --2- 172.21.15.82:0/283467393 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d00d48e0 0x7fa8d00ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.024+0000 7fa8cfe00640 1 --2- 172.21.15.82:0/283467393 >> v2:172.21.15.90:3300/0 conn(0x7fa8d00d2ab0 0x7fa8d015e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.024+0000 7fa8cf400640 1 -- 172.21.15.82:0/283467393 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202284581 0 0) 0x7fa8d00b33c0 con 0x7fa8d015e040 2024-03-16T17:58:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.024+0000 7fa8cf400640 1 -- 172.21.15.82:0/283467393 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b4003520 con 0x7fa8d015e040 2024-03-16T17:58:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.024+0000 7fa8d5c00640 1 -- 172.21.15.82:0/283467393 >> v2:172.21.15.90:3300/0 conn(0x7fa8d00d2ab0 msgr2=0x7fa8d015e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:15.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.024+0000 7fa8d5c00640 1 --2- 172.21.15.82:0/283467393 >> v2:172.21.15.90:3300/0 conn(0x7fa8d00d2ab0 0x7fa8d015e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.024+0000 7fa8d5c00640 1 -- 172.21.15.82:0/283467393 >> v1:172.21.15.82:6789/0 conn(0x7fa8d015e040 legacy=0x7fa8d00d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.024+0000 7fa8d5c00640 1 -- 172.21.15.82:0/283467393 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00b3610 con 0x7fa8d00d48e0 2024-03-16T17:58:15.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.026+0000 7fa8d5c00640 1 --2- 172.21.15.82:0/283467393 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d00d48e0 0x7fa8d00ce9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80097a0 tx=0x7fa8c802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2666977399288c93 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.027+0000 7fa8cf400640 1 -- 172.21.15.82:0/283467393 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8031780 con 0x7fa8d00d48e0 2024-03-16T17:58:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.027+0000 7fa8cf400640 1 -- 172.21.15.82:0/283467393 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c80318e0 con 0x7fa8d00d48e0 2024-03-16T17:58:15.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.027+0000 7fa8cf400640 1 -- 172.21.15.82:0/283467393 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8031c70 con 0x7fa8d00d48e0 2024-03-16T17:58:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.027+0000 7fa8d7c00640 1 -- 172.21.15.82:0/283467393 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d00d48e0 msgr2=0x7fa8d00ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.027+0000 7fa8d7c00640 1 --2- 172.21.15.82:0/283467393 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d00d48e0 0x7fa8d00ce9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80097a0 tx=0x7fa8c802b820 comp rx=0 tx=0).stop 2024-03-16T17:58:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.027+0000 7fa8d7c00640 1 -- 172.21.15.82:0/283467393 shutdown_connections 2024-03-16T17:58:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.027+0000 7fa8d7c00640 1 --2- 172.21.15.82:0/283467393 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d00d48e0 0x7fa8d00ce9d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.027+0000 7fa8d7c00640 1 --2- 172.21.15.82:0/283467393 >> v2:172.21.15.90:3300/0 conn(0x7fa8d00d2ab0 0x7fa8d015e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.027+0000 7fa8d7c00640 1 -- 172.21.15.82:0/283467393 >> 172.21.15.82:0/283467393 conn(0x7fa8d00d3dc0 msgr2=0x7fa8d00d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:15.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.027+0000 7fa8d7c00640 1 -- 172.21.15.82:0/283467393 shutdown_connections 2024-03-16T17:58:15.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.028+0000 7fa8d7c00640 1 -- 172.21.15.82:0/283467393 wait complete. 2024-03-16T17:58:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.029+0000 7fa8d7c00640 1 Processor -- start 2024-03-16T17:58:15.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.029+0000 7fa8d7c00640 1 -- start start 2024-03-16T17:58:15.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.030+0000 7fa8d7c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8d00d2ab0 0x7fa8d00a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.030+0000 7fa8d7c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d00d48e0 0x7fa8d00cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.030+0000 7fa8d7c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d015e040 0x7fa8d0151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.030+0000 7fa8d7c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00cbac0 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.030+0000 7fa8d7c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00a0ea0 con 0x7fa8d00d48e0 2024-03-16T17:58:15.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.030+0000 7fa8d7c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0151770 con 0x7fa8d015e040 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.030+0000 7fa8d5200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8d00d2ab0 0x7fa8d00a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8d5200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8d00d2ab0 0x7fa8d00a1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46986/0 (socket says 172.21.15.82:46986) 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8d5200640 1 -- 172.21.15.82:0/2665191708 learned_addr learned my addr 172.21.15.82:0/2665191708 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8cfe00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d00d48e0 0x7fa8d00cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8d5200640 1 -- 172.21.15.82:0/2665191708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d015e040 msgr2=0x7fa8d0151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8d5200640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d015e040 0x7fa8d0151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8d5200640 1 -- 172.21.15.82:0/2665191708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d00d48e0 msgr2=0x7fa8d00cb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8d5200640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d00d48e0 0x7fa8d00cb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8d5200640 1 -- 172.21.15.82:0/2665191708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c8009400 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8cfe00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d00d48e0 0x7fa8d00cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8d5200640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8d00d2ab0 0x7fa8d00a1330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000cad0 tx=0x7fa8c000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.031+0000 7fa8ccc00640 1 -- 172.21.15.82:0/2665191708 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c000ac30 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.032+0000 7fa8ccc00640 1 -- 172.21.15.82:0/2665191708 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c000ad90 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.032+0000 7fa8d7c00640 1 -- 172.21.15.82:0/2665191708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d00a0290 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.032+0000 7fa8ccc00640 1 -- 172.21.15.82:0/2665191708 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0012620 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.032+0000 7fa8d7c00640 1 -- 172.21.15.82:0/2665191708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d00a0850 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.033+0000 7fa8d7c00640 1 -- 172.21.15.82:0/2665191708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d0067f30 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.034+0000 7fa8ccc00640 1 -- 172.21.15.82:0/2665191708 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa8c0007070 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.034+0000 7fa8ccc00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b3b0 0x7fa8ac06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.035+0000 7fa8ccc00640 1 -- 172.21.15.82:0/2665191708 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8c0089bb0 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.035+0000 7fa8cfe00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b3b0 0x7fa8ac06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.035+0000 7fa8cfe00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b3b0 0x7fa8ac06d870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa8d01f6ee0 tx=0x7fa8b8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:15.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.039+0000 7fa8ccc00640 1 -- 172.21.15.82:0/2665191708 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa8c0054810 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.236+0000 7fa8d7c00640 1 -- 172.21.15.82:0/2665191708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8d015c8e0 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.236+0000 7fa8ccc00640 1 -- 172.21.15.82:0/2665191708 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa8c00170a0 con 0x7fa8d00d2ab0 2024-03-16T17:58:15.240 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:58:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.239+0000 7fa8d7c00640 1 -- 172.21.15.82:0/2665191708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b3b0 msgr2=0x7fa8ac06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.239+0000 7fa8d7c00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b3b0 0x7fa8ac06d870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa8d01f6ee0 tx=0x7fa8b8008040 comp rx=0 tx=0).stop 2024-03-16T17:58:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.239+0000 7fa8d7c00640 1 -- 172.21.15.82:0/2665191708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8d00d2ab0 msgr2=0x7fa8d00a1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.239+0000 7fa8d7c00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8d00d2ab0 0x7fa8d00a1330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000cad0 tx=0x7fa8c000cf90 comp rx=0 tx=0).stop 2024-03-16T17:58:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.239+0000 7fa8d7c00640 1 -- 172.21.15.82:0/2665191708 shutdown_connections 2024-03-16T17:58:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.239+0000 7fa8d7c00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa8ac06b3b0 0x7fa8ac06d870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.239+0000 7fa8d7c00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8d015e040 0x7fa8d0151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.239+0000 7fa8d7c00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8d00d48e0 0x7fa8d00cb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.239+0000 7fa8d7c00640 1 --2- 172.21.15.82:0/2665191708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8d00d2ab0 0x7fa8d00a1330 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.239+0000 7fa8d7c00640 1 -- 172.21.15.82:0/2665191708 >> 172.21.15.82:0/2665191708 conn(0x7fa8d00d3dc0 msgr2=0x7fa8d00d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.240+0000 7fa8d7c00640 1 -- 172.21.15.82:0/2665191708 shutdown_connections 2024-03-16T17:58:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.240+0000 7fa8d7c00640 1 -- 172.21.15.82:0/2665191708 wait complete. 2024-03-16T17:58:15.262 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:58:15.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.470+0000 7fb129e00640 1 Processor -- start 2024-03-16T17:58:15.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.471+0000 7fb129e00640 1 -- start start 2024-03-16T17:58:15.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.472+0000 7fb129e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb1240d2ab0 0x7fb12415e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.472+0000 7fb129e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb1240d48e0 0x7fb1240ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.472+0000 7fb129e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1240b33c0 con 0x7fb12415e040 2024-03-16T17:58:15.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.472+0000 7fb129e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1240d1cd0 con 0x7fb1240d48e0 2024-03-16T17:58:15.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.472+0000 7fb129e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1240d1fb0 con 0x7fb1240d2ab0 2024-03-16T17:58:15.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.472+0000 7fb123400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb1240d48e0 0x7fb1240ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.473+0000 7fb123400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb1240d48e0 0x7fb1240ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46988/0 (socket says 172.21.15.82:46988) 2024-03-16T17:58:15.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.473+0000 7fb123400640 1 -- 172.21.15.82:0/892794934 learned_addr learned my addr 172.21.15.82:0/892794934 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:15.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.473+0000 7fb122000640 1 --2- 172.21.15.82:0/892794934 >> v2:172.21.15.90:3300/0 conn(0x7fb1240d2ab0 0x7fb12415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.473+0000 7fb123400640 1 -- 172.21.15.82:0/892794934 >> v2:172.21.15.90:3300/0 conn(0x7fb1240d2ab0 msgr2=0x7fb12415e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:15.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.473+0000 7fb123400640 1 --2- 172.21.15.82:0/892794934 >> v2:172.21.15.90:3300/0 conn(0x7fb1240d2ab0 0x7fb12415e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.473+0000 7fb123400640 1 -- 172.21.15.82:0/892794934 >> v1:172.21.15.90:6789/0 conn(0x7fb12415e040 legacy=0x7fb1240d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:15.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.473+0000 7fb123400640 1 -- 172.21.15.82:0/892794934 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240b3610 con 0x7fb1240d48e0 2024-03-16T17:58:15.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.474+0000 7fb123400640 1 --2- 172.21.15.82:0/892794934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d48e0 0x7fb1240ce9d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb1240ce3a0 tx=0x7fb11c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409695e3ba3c071c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:15.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.474+0000 7fb121600640 1 -- 172.21.15.82:0/892794934 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c02c9f0 con 0x7fb1240d48e0 2024-03-16T17:58:15.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.474+0000 7fb121600640 1 -- 172.21.15.82:0/892794934 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c02cb50 con 0x7fb1240d48e0 2024-03-16T17:58:15.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.474+0000 7fb121600640 1 -- 172.21.15.82:0/892794934 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c0343f0 con 0x7fb1240d48e0 2024-03-16T17:58:15.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.475+0000 7fb129e00640 1 -- 172.21.15.82:0/892794934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d48e0 msgr2=0x7fb1240ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.475+0000 7fb129e00640 1 --2- 172.21.15.82:0/892794934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d48e0 0x7fb1240ce9d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb1240ce3a0 tx=0x7fb11c02be50 comp rx=0 tx=0).stop 2024-03-16T17:58:15.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.475+0000 7fb129e00640 1 -- 172.21.15.82:0/892794934 shutdown_connections 2024-03-16T17:58:15.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.475+0000 7fb129e00640 1 --2- 172.21.15.82:0/892794934 >> v2:172.21.15.90:3300/0 conn(0x7fb1240d2ab0 0x7fb12415e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.475+0000 7fb129e00640 1 --2- 172.21.15.82:0/892794934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d48e0 0x7fb1240ce9d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.475+0000 7fb129e00640 1 -- 172.21.15.82:0/892794934 >> 172.21.15.82:0/892794934 conn(0x7fb1240d3dc0 msgr2=0x7fb1240d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:15.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.475+0000 7fb129e00640 1 -- 172.21.15.82:0/892794934 shutdown_connections 2024-03-16T17:58:15.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.475+0000 7fb129e00640 1 -- 172.21.15.82:0/892794934 wait complete. 2024-03-16T17:58:15.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.477+0000 7fb129e00640 1 Processor -- start 2024-03-16T17:58:15.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.477+0000 7fb129e00640 1 -- start start 2024-03-16T17:58:15.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.478+0000 7fb129e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d2ab0 0x7fb1240ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.478+0000 7fb129e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1240d48e0 0x7fb1240cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.478+0000 7fb129e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb12415e040 0x7fb1240cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.478+0000 7fb129e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1241f7900 con 0x7fb1240d2ab0 2024-03-16T17:58:15.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.478+0000 7fb129e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1241f37f0 con 0x7fb12415e040 2024-03-16T17:58:15.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.478+0000 7fb129e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1241f3ad0 con 0x7fb1240d48e0 2024-03-16T17:58:15.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.478+0000 7fb122a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d2ab0 0x7fb1240ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.478+0000 7fb122a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d2ab0 0x7fb1240ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47004/0 (socket says 172.21.15.82:47004) 2024-03-16T17:58:15.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.478+0000 7fb122a00640 1 -- 172.21.15.82:0/1197912147 learned_addr learned my addr 172.21.15.82:0/1197912147 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb123400640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb12415e040 0x7fb1240cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb122000640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1240d48e0 0x7fb1240cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb122a00640 1 -- 172.21.15.82:0/1197912147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1240d48e0 msgr2=0x7fb1240cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb122a00640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1240d48e0 0x7fb1240cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb122a00640 1 -- 172.21.15.82:0/1197912147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb12415e040 msgr2=0x7fb1240cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb122a00640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb12415e040 0x7fb1240cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb122a00640 1 -- 172.21.15.82:0/1197912147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c009b30 con 0x7fb1240d2ab0 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb122a00640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d2ab0 0x7fb1240ca060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb11400b3f0 tx=0x7fb11400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb11b400640 1 -- 172.21.15.82:0/1197912147 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114011830 con 0x7fb1240d2ab0 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb11b400640 1 -- 172.21.15.82:0/1197912147 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb114011990 con 0x7fb1240d2ab0 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb11b400640 1 -- 172.21.15.82:0/1197912147 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114011d40 con 0x7fb1240d2ab0 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.479+0000 7fb129e00640 1 -- 172.21.15.82:0/1197912147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1241f3db0 con 0x7fb1240d2ab0 2024-03-16T17:58:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.480+0000 7fb129e00640 1 -- 172.21.15.82:0/1197912147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1240ca5a0 con 0x7fb1240d2ab0 2024-03-16T17:58:15.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.481+0000 7fb129e00640 1 -- 172.21.15.82:0/1197912147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1241f3fa0 con 0x7fb1240d2ab0 2024-03-16T17:58:15.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.482+0000 7fb11b400640 1 -- 172.21.15.82:0/1197912147 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fb11401a400 con 0x7fb1240d2ab0 2024-03-16T17:58:15.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.483+0000 7fb11b400640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb10006b3b0 0x7fb10006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.483+0000 7fb11b400640 1 -- 172.21.15.82:0/1197912147 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb114089a40 con 0x7fb1240d2ab0 2024-03-16T17:58:15.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.483+0000 7fb122000640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb10006b3b0 0x7fb10006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.484+0000 7fb122000640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb10006b3b0 0x7fb10006d870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1240a1dc0 tx=0x7fb10c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:15.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.487+0000 7fb11b400640 1 -- 172.21.15.82:0/1197912147 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb11408be60 con 0x7fb1240d2ab0 2024-03-16T17:58:15.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.676+0000 7fb129e00640 1 -- 172.21.15.82:0/1197912147 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1240b4da0 con 0x7fb1240d2ab0 2024-03-16T17:58:15.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.677+0000 7fb11b400640 1 -- 172.21.15.82:0/1197912147 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1140150a0 con 0x7fb1240d2ab0 2024-03-16T17:58:15.681 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:58:15.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 -- 172.21.15.82:0/1197912147 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb10006b3b0 msgr2=0x7fb10006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb10006b3b0 0x7fb10006d870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1240a1dc0 tx=0x7fb10c008040 comp rx=0 tx=0).stop 2024-03-16T17:58:15.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 -- 172.21.15.82:0/1197912147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d2ab0 msgr2=0x7fb1240ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d2ab0 0x7fb1240ca060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb11400b3f0 tx=0x7fb11400b8b0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 -- 172.21.15.82:0/1197912147 shutdown_connections 2024-03-16T17:58:15.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb10006b3b0 0x7fb10006d870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb12415e040 0x7fb1240cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb1240d48e0 0x7fb1240cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 --2- 172.21.15.82:0/1197912147 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1240d2ab0 0x7fb1240ca060 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 -- 172.21.15.82:0/1197912147 >> 172.21.15.82:0/1197912147 conn(0x7fb1240d3dc0 msgr2=0x7fb1240ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:15.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 -- 172.21.15.82:0/1197912147 shutdown_connections 2024-03-16T17:58:15.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.680+0000 7fb129e00640 1 -- 172.21.15.82:0/1197912147 wait complete. 2024-03-16T17:58:15.701 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:15.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.905+0000 7f3c9da00640 1 Processor -- start 2024-03-16T17:58:15.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.906+0000 7f3c9da00640 1 -- start start 2024-03-16T17:58:15.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.907+0000 7f3c9da00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3c9806e2d0 0x7f3c9815af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.907+0000 7f3c9da00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c980cf6d0 0x7f3c980cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.907+0000 7f3c9da00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9815e4a0 con 0x7f3c9806e910 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.907+0000 7f3c9da00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c980d00a0 con 0x7f3c980cf6d0 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.907+0000 7f3c9da00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9815e780 con 0x7f3c9806e2d0 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.908+0000 7f3c9d000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c980cf6d0 0x7f3c980cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.908+0000 7f3c9d000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c980cf6d0 0x7f3c980cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47012/0 (socket says 172.21.15.82:47012) 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.908+0000 7f3c9d000640 1 -- 172.21.15.82:0/2979512228 learned_addr learned my addr 172.21.15.82:0/2979512228 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.908+0000 7f3c8f400640 1 --2- 172.21.15.82:0/2979512228 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3c9806e2d0 0x7f3c9815af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.908+0000 7f3c9d000640 1 -- 172.21.15.82:0/2979512228 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3c9806e2d0 msgr2=0x7f3c9815af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.908+0000 7f3c9d000640 1 --2- 172.21.15.82:0/2979512228 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3c9806e2d0 0x7f3c9815af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.908+0000 7f3c9d000640 1 -- 172.21.15.82:0/2979512228 >> v1:172.21.15.90:6789/0 conn(0x7f3c9806e910 legacy=0x7f3c9815da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:15.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.908+0000 7f3c9d000640 1 -- 172.21.15.82:0/2979512228 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9815a720 con 0x7f3c980cf6d0 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.909+0000 7f3c9d000640 1 --2- 172.21.15.82:0/2979512228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c980cf6d0 0x7f3c980cfb60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009e50 tx=0x7f3c9002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10980e7f9b5fc207 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.909+0000 7f3c97400640 1 -- 172.21.15.82:0/2979512228 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c9002c9e0 con 0x7f3c980cf6d0 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.909+0000 7f3c97400640 1 -- 172.21.15.82:0/2979512228 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c9002cb40 con 0x7f3c980cf6d0 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.910+0000 7f3c97400640 1 -- 172.21.15.82:0/2979512228 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c900343f0 con 0x7f3c980cf6d0 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.910+0000 7f3c9da00640 1 -- 172.21.15.82:0/2979512228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c980cf6d0 msgr2=0x7f3c980cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.910+0000 7f3c9da00640 1 --2- 172.21.15.82:0/2979512228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c980cf6d0 0x7f3c980cfb60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3c90009e50 tx=0x7f3c9002be40 comp rx=0 tx=0).stop 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.910+0000 7f3c9da00640 1 -- 172.21.15.82:0/2979512228 shutdown_connections 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.910+0000 7f3c9da00640 1 --2- 172.21.15.82:0/2979512228 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3c9806e2d0 0x7f3c9815af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.910+0000 7f3c9da00640 1 --2- 172.21.15.82:0/2979512228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c980cf6d0 0x7f3c980cfb60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.910+0000 7f3c9da00640 1 -- 172.21.15.82:0/2979512228 >> 172.21.15.82:0/2979512228 conn(0x7f3c9806ae70 msgr2=0x7f3c9806b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.910+0000 7f3c9da00640 1 -- 172.21.15.82:0/2979512228 shutdown_connections 2024-03-16T17:58:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.910+0000 7f3c9da00640 1 -- 172.21.15.82:0/2979512228 wait complete. 2024-03-16T17:58:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.912+0000 7f3c9da00640 1 Processor -- start 2024-03-16T17:58:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.912+0000 7f3c9da00640 1 -- start start 2024-03-16T17:58:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.913+0000 7f3c9da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c9806e2d0 0x7f3c981f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.913+0000 7f3c9da00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3c9806e910 0x7f3c98151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.913+0000 7f3c9da00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c980cf6d0 0x7f3c98152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.913+0000 7f3c9da00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98154e90 con 0x7f3c9806e2d0 2024-03-16T17:58:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.913+0000 7f3c9da00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98155170 con 0x7f3c980cf6d0 2024-03-16T17:58:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.913+0000 7f3c9da00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c98155450 con 0x7f3c9806e910 2024-03-16T17:58:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.913+0000 7f3c97e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c9806e2d0 0x7f3c981f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.913+0000 7f3c97e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c9806e2d0 0x7f3c981f2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47018/0 (socket says 172.21.15.82:47018) 2024-03-16T17:58:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.913+0000 7f3c97e00640 1 -- 172.21.15.82:0/3672890164 learned_addr learned my addr 172.21.15.82:0/3672890164 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c8f400640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3c9806e910 0x7f3c98151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c9d000640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c980cf6d0 0x7f3c98152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c97e00640 1 -- 172.21.15.82:0/3672890164 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3c9806e910 msgr2=0x7f3c98151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c97e00640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3c9806e910 0x7f3c98151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c97e00640 1 -- 172.21.15.82:0/3672890164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c980cf6d0 msgr2=0x7f3c98152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c97e00640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c980cf6d0 0x7f3c98152000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c97e00640 1 -- 172.21.15.82:0/3672890164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c90009b30 con 0x7f3c9806e2d0 2024-03-16T17:58:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c97e00640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c9806e2d0 0x7f3c981f2230 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800c430 tx=0x7f3c8800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c94c00640 1 -- 172.21.15.82:0/3672890164 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88011790 con 0x7f3c9806e2d0 2024-03-16T17:58:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c94c00640 1 -- 172.21.15.82:0/3672890164 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c880118f0 con 0x7f3c9806e2d0 2024-03-16T17:58:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c94c00640 1 -- 172.21.15.82:0/3672890164 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c88011ce0 con 0x7f3c9806e2d0 2024-03-16T17:58:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c9da00640 1 -- 172.21.15.82:0/3672890164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c98155730 con 0x7f3c9806e2d0 2024-03-16T17:58:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.914+0000 7f3c9da00640 1 -- 172.21.15.82:0/3672890164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c98154b20 con 0x7f3c9806e2d0 2024-03-16T17:58:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.915+0000 7f3c9da00640 1 -- 172.21.15.82:0/3672890164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c980cd890 con 0x7f3c9806e2d0 2024-03-16T17:58:15.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.916+0000 7f3c94c00640 1 -- 172.21.15.82:0/3672890164 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f3c8801c460 con 0x7f3c9806e2d0 2024-03-16T17:58:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.917+0000 7f3c94c00640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3c7406b3b0 0x7f3c7406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.917+0000 7f3c94c00640 1 -- 172.21.15.82:0/3672890164 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c88089c00 con 0x7f3c9806e2d0 2024-03-16T17:58:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.917+0000 7f3c8f400640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3c7406b3b0 0x7f3c7406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:15.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.918+0000 7f3c8f400640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3c7406b3b0 0x7f3c7406d870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3c80006fd0 tx=0x7f3c80008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:15.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:15.923+0000 7f3c94c00640 1 -- 172.21.15.82:0/3672890164 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3c88090080 con 0x7f3c9806e2d0 2024-03-16T17:58:16.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.114+0000 7f3c9da00640 1 -- 172.21.15.82:0/3672890164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c980b14d0 con 0x7f3c9806e2d0 2024-03-16T17:58:16.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.117+0000 7f3c94c00640 1 -- 172.21.15.82:0/3672890164 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f3c88054860 con 0x7f3c9806e2d0 2024-03-16T17:58:16.122 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:16.122 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:16.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 -- 172.21.15.82:0/3672890164 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3c7406b3b0 msgr2=0x7f3c7406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3c7406b3b0 0x7f3c7406d870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3c80006fd0 tx=0x7f3c80008040 comp rx=0 tx=0).stop 2024-03-16T17:58:16.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 -- 172.21.15.82:0/3672890164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c9806e2d0 msgr2=0x7f3c981f2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c9806e2d0 0x7f3c981f2230 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3c8800c430 tx=0x7f3c8800c8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 -- 172.21.15.82:0/3672890164 shutdown_connections 2024-03-16T17:58:16.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3c7406b3b0 0x7f3c7406d870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3c980cf6d0 0x7f3c98152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3c9806e910 0x7f3c98151ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 --2- 172.21.15.82:0/3672890164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c9806e2d0 0x7f3c981f2230 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 -- 172.21.15.82:0/3672890164 >> 172.21.15.82:0/3672890164 conn(0x7f3c9806ae70 msgr2=0x7f3c98154310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:16.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 -- 172.21.15.82:0/3672890164 shutdown_connections 2024-03-16T17:58:16.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.120+0000 7f3c9da00640 1 -- 172.21.15.82:0/3672890164 wait complete. 2024-03-16T17:58:16.142 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:16.143 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:16.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.339+0000 7f7ec2200640 1 Processor -- start 2024-03-16T17:58:16.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.339+0000 7f7ec2200640 1 -- start start 2024-03-16T17:58:16.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.341+0000 7f7ec2200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7ebc15fcf0 0x7f7ebc165e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.341+0000 7f7ec2200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc166340 0x7f7ebc06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.341+0000 7f7ec2200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc0b29b0 con 0x7f7ebc06de40 2024-03-16T17:58:16.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.341+0000 7f7ec2200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ebc06e2d0 con 0x7f7ebc15fcf0 2024-03-16T17:58:16.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.341+0000 7f7ec2200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0b2f70 con 0x7f7ebc166340 2024-03-16T17:58:16.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.341+0000 7f7ec1800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7ebc06de40 0x7f7ebc0711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47604/0 (socket says 172.21.15.82:47604) 2024-03-16T17:58:16.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.341+0000 7f7ec1800640 1 -- 172.21.15.82:0/4159126907 learned_addr learned my addr 172.21.15.82:0/4159126907 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.341+0000 7f7ec0e00640 1 --2- 172.21.15.82:0/4159126907 >> v2:172.21.15.90:3300/0 conn(0x7f7ebc15fcf0 0x7f7ebc165e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.341+0000 7f7ebbe00640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc166340 0x7f7ebc06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053517732 0 0) 0x7f7ebc0b29b0 con 0x7f7ebc06de40 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea4003520 con 0x7f7ebc06de40 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3354174156 0 0) 0x7f7ea4003520 con 0x7f7ebc06de40 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc166340 msgr2=0x7f7ebc06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc166340 0x7f7ebc06d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 >> v2:172.21.15.90:3300/0 conn(0x7f7ebc15fcf0 msgr2=0x7f7ebc165e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 --2- 172.21.15.82:0/4159126907 >> v2:172.21.15.90:3300/0 conn(0x7f7ebc15fcf0 0x7f7ebc165e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc1668a0 con 0x7f7ebc06de40 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f7eb40019f0 con 0x7f7ebc06de40 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 >> v1:172.21.15.82:6789/0 conn(0x7f7ebc06de40 legacy=0x7f7ebc0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea40039e0 0x7f7ea4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc166340 0x7f7ea40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ea4004d00 0x7f7ea4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc06e2d0 con 0x7f7ea4004d00 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0b2f70 con 0x7f7ea40039e0 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.342+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea400ba30 con 0x7f7ebc166340 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.343+0000 7f7ec1800640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ea4004d00 0x7f7ea4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.343+0000 7f7ec0e00640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea40039e0 0x7f7ea4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.343+0000 7f7ebbe00640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc166340 0x7f7ea40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.343+0000 7f7ec1800640 1 -- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc166340 msgr2=0x7f7ea40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.343+0000 7f7ec1800640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc166340 0x7f7ea40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.343+0000 7f7ec1800640 1 -- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea40039e0 msgr2=0x7f7ea4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.343+0000 7f7ec1800640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea40039e0 0x7f7ea4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.343+0000 7f7ec1800640 1 -- 172.21.15.82:0/4159126907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea4003520 con 0x7f7ea4004d00 2024-03-16T17:58:16.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.344+0000 7f7ec1800640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ea4004d00 0x7f7ea4005180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400a760 tx=0x7f7ea40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6b2f71da436cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:16.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.344+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4034630 con 0x7f7ea4004d00 2024-03-16T17:58:16.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.344+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb402f070 con 0x7f7ea4004d00 2024-03-16T17:58:16.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.344+0000 7f7ebb400640 1 -- 172.21.15.82:0/4159126907 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb40349f0 con 0x7f7ea4004d00 2024-03-16T17:58:16.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.345+0000 7f7ec2200640 1 -- 172.21.15.82:0/4159126907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ea4004d00 msgr2=0x7f7ea4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.345+0000 7f7ec2200640 1 --2- 172.21.15.82:0/4159126907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ea4004d00 0x7f7ea4005180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400a760 tx=0x7f7ea40057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.345+0000 7f7ec1800640 1 -- 172.21.15.82:0/4159126907 reap_dead start 2024-03-16T17:58:16.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.345+0000 7f7ec2200640 1 -- 172.21.15.82:0/4159126907 shutdown_connections 2024-03-16T17:58:16.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.345+0000 7f7ec2200640 1 -- 172.21.15.82:0/4159126907 >> 172.21.15.82:0/4159126907 conn(0x7f7ebc06ae70 msgr2=0x7f7ebc06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:16.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.345+0000 7f7ec2200640 1 -- 172.21.15.82:0/4159126907 shutdown_connections 2024-03-16T17:58:16.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.345+0000 7f7ec2200640 1 -- 172.21.15.82:0/4159126907 wait complete. 2024-03-16T17:58:16.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.347+0000 7f7ec2200640 1 Processor -- start 2024-03-16T17:58:16.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.347+0000 7f7ec2200640 1 -- start start 2024-03-16T17:58:16.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec2200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ebc1479f0 0x7f7ebc0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec2200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc0ccc60 0x7f7ebc159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec2200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ebc15aad0 0x7f7ebc15af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec2200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc15b4a0 con 0x7f7ebc1479f0 2024-03-16T17:58:16.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec2200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0cdd30 con 0x7f7ebc15aad0 2024-03-16T17:58:16.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec2200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0cd120 con 0x7f7ebc0ccc60 2024-03-16T17:58:16.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec0e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ebc1479f0 0x7f7ebc0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec0e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ebc1479f0 0x7f7ebc0ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47024/0 (socket says 172.21.15.82:47024) 2024-03-16T17:58:16.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec0e00640 1 -- 172.21.15.82:0/648772967 learned_addr learned my addr 172.21.15.82:0/648772967 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:16.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ebbe00640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc0ccc60 0x7f7ebc159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec1800640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ebc15aad0 0x7f7ebc15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.348+0000 7f7ec0e00640 1 -- 172.21.15.82:0/648772967 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc0ccc60 msgr2=0x7f7ebc159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.349+0000 7f7ec0e00640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc0ccc60 0x7f7ebc159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.349+0000 7f7ec0e00640 1 -- 172.21.15.82:0/648772967 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ebc15aad0 msgr2=0x7f7ebc15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.349+0000 7f7ec0e00640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ebc15aad0 0x7f7ebc15af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.349+0000 7f7ec0e00640 1 -- 172.21.15.82:0/648772967 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc0cd400 con 0x7f7ebc1479f0 2024-03-16T17:58:16.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.349+0000 7f7ec0e00640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ebc1479f0 0x7f7ebc0ce1c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0011700 tx=0x7f7eb0011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:16.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.349+0000 7f7eb8c00640 1 -- 172.21.15.82:0/648772967 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb000f9c0 con 0x7f7ebc1479f0 2024-03-16T17:58:16.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.349+0000 7f7eb8c00640 1 -- 172.21.15.82:0/648772967 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb000fb20 con 0x7f7ebc1479f0 2024-03-16T17:58:16.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.349+0000 7f7eb8c00640 1 -- 172.21.15.82:0/648772967 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb000fe90 con 0x7f7ebc1479f0 2024-03-16T17:58:16.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.349+0000 7f7ec2200640 1 -- 172.21.15.82:0/648772967 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc0cd6e0 con 0x7f7ebc1479f0 2024-03-16T17:58:16.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.349+0000 7f7ec2200640 1 -- 172.21.15.82:0/648772967 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc147dd0 con 0x7f7ebc1479f0 2024-03-16T17:58:16.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.351+0000 7f7ec2200640 1 -- 172.21.15.82:0/648772967 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc04f250 con 0x7f7ebc1479f0 2024-03-16T17:58:16.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.351+0000 7f7eb8c00640 1 -- 172.21.15.82:0/648772967 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f7eb0015460 con 0x7f7ebc1479f0 2024-03-16T17:58:16.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.352+0000 7f7eb8c00640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7e9c06b3e0 0x7f7e9c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.352+0000 7f7eb8c00640 1 -- 172.21.15.82:0/648772967 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7eb008a150 con 0x7f7ebc1479f0 2024-03-16T17:58:16.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.353+0000 7f7ebbe00640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7e9c06b3e0 0x7f7e9c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.353+0000 7f7ebbe00640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7e9c06b3e0 0x7f7e9c06d8a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc1fb760 tx=0x7f7ea8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:16.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.357+0000 7f7eb8c00640 1 -- 172.21.15.82:0/648772967 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7eb0054e60 con 0x7f7ebc1479f0 2024-03-16T17:58:16.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.553+0000 7f7ec2200640 1 -- 172.21.15.82:0/648772967 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ebc06e670 con 0x7f7ebc1479f0 2024-03-16T17:58:16.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.554+0000 7f7eb8c00640 1 -- 172.21.15.82:0/648772967 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7eb0058520 con 0x7f7ebc1479f0 2024-03-16T17:58:16.558 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 -- 172.21.15.82:0/648772967 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7e9c06b3e0 msgr2=0x7f7e9c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7e9c06b3e0 0x7f7e9c06d8a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc1fb760 tx=0x7f7ea8008040 comp rx=0 tx=0).stop 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 -- 172.21.15.82:0/648772967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ebc1479f0 msgr2=0x7f7ebc0ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ebc1479f0 0x7f7ebc0ce1c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0011700 tx=0x7f7eb0011bc0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 -- 172.21.15.82:0/648772967 shutdown_connections 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7e9c06b3e0 0x7f7e9c06d8a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ebc15aad0 0x7f7ebc15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ebc0ccc60 0x7f7ebc159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 --2- 172.21.15.82:0/648772967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ebc1479f0 0x7f7ebc0ce1c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 -- 172.21.15.82:0/648772967 >> 172.21.15.82:0/648772967 conn(0x7f7ebc06ae70 msgr2=0x7f7ebc15ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:16.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 -- 172.21.15.82:0/648772967 shutdown_connections 2024-03-16T17:58:16.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.557+0000 7f7ec2200640 1 -- 172.21.15.82:0/648772967 wait complete. 2024-03-16T17:58:16.579 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:58:16.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.787+0000 7f8448e00640 1 Processor -- start 2024-03-16T17:58:16.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.787+0000 7f8448e00640 1 -- start start 2024-03-16T17:58:16.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.789+0000 7f8448e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 0x7f8444161f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.789+0000 7f8448e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f844406bf80 0x7f844406c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.789+0000 7f8448e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8444162990 con 0x7f844406fd10 2024-03-16T17:58:16.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.789+0000 7f8448e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8444162480 con 0x7f844406bf80 2024-03-16T17:58:16.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.789+0000 7f8448e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844415bf70 con 0x7f8444161b60 2024-03-16T17:58:16.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.789+0000 7f8442800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f844406bf80 0x7f844406c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.789+0000 7f8442800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f844406bf80 0x7f844406c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47030/0 (socket says 172.21.15.82:47030) 2024-03-16T17:58:16.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.789+0000 7f8442800640 1 -- 172.21.15.82:0/419692651 learned_addr learned my addr 172.21.15.82:0/419692651 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:16.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.789+0000 7f8441e00640 1 --2- 172.21.15.82:0/419692651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 0x7f8444161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.790+0000 7f8442800640 1 -- 172.21.15.82:0/419692651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 msgr2=0x7f8444161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.790+0000 7f8442800640 1 --2- 172.21.15.82:0/419692651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 0x7f8444161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.790+0000 7f8442800640 1 -- 172.21.15.82:0/419692651 >> v1:172.21.15.90:6789/0 conn(0x7f844406fd10 legacy=0x7f84440d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.790+0000 7f8442800640 1 -- 172.21.15.82:0/419692651 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844406ec70 con 0x7f844406bf80 2024-03-16T17:58:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.790+0000 7f8441e00640 1 --2- 172.21.15.82:0/419692651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 0x7f8444161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.791+0000 7f8442800640 1 --2- 172.21.15.82:0/419692651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 0x7f844406c3d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f843c009e50 tx=0x7f843c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45291ed208ddfc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:16.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.791+0000 7f8440a00640 1 -- 172.21.15.82:0/419692651 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c02ca70 con 0x7f844406bf80 2024-03-16T17:58:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.791+0000 7f8440a00640 1 -- 172.21.15.82:0/419692651 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c02cbd0 con 0x7f844406bf80 2024-03-16T17:58:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.791+0000 7f8440a00640 1 -- 172.21.15.82:0/419692651 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c0343f0 con 0x7f844406bf80 2024-03-16T17:58:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.791+0000 7f8448e00640 1 -- 172.21.15.82:0/419692651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 msgr2=0x7f844406c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.791+0000 7f8448e00640 1 --2- 172.21.15.82:0/419692651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 0x7f844406c3d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f843c009e50 tx=0x7f843c02bed0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.792+0000 7f8448e00640 1 -- 172.21.15.82:0/419692651 shutdown_connections 2024-03-16T17:58:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.792+0000 7f8448e00640 1 --2- 172.21.15.82:0/419692651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 0x7f844406c3d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.792+0000 7f8448e00640 1 --2- 172.21.15.82:0/419692651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 0x7f8444161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.792+0000 7f8448e00640 1 -- 172.21.15.82:0/419692651 >> 172.21.15.82:0/419692651 conn(0x7f84440d1bd0 msgr2=0x7f84440d1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.792+0000 7f8448e00640 1 -- 172.21.15.82:0/419692651 shutdown_connections 2024-03-16T17:58:16.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.792+0000 7f8448e00640 1 -- 172.21.15.82:0/419692651 wait complete. 2024-03-16T17:58:16.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.793+0000 7f8448e00640 1 Processor -- start 2024-03-16T17:58:16.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.793+0000 7f8448e00640 1 -- start start 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.794+0000 7f8448e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 0x7f8444151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.794+0000 7f8448e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f844406fd10 0x7f84441e8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.794+0000 7f8448e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 0x7f844415e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.794+0000 7f8448e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84440c8190 con 0x7f844406bf80 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.794+0000 7f8448e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8444152040 con 0x7f844406fd10 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.794+0000 7f8448e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84440c70f0 con 0x7f8444161b60 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8441e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 0x7f8444151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8441e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 0x7f8444151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47042/0 (socket says 172.21.15.82:47042) 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8441e00640 1 -- 172.21.15.82:0/580963037 learned_addr learned my addr 172.21.15.82:0/580963037 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8442800640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 0x7f844415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8441400640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f844406fd10 0x7f84441e8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8441e00640 1 -- 172.21.15.82:0/580963037 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 msgr2=0x7f844415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8441e00640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 0x7f844415e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8441e00640 1 -- 172.21.15.82:0/580963037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f844406fd10 msgr2=0x7f84441e8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8441e00640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f844406fd10 0x7f84441e8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8441e00640 1 -- 172.21.15.82:0/580963037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c009b30 con 0x7f844406bf80 2024-03-16T17:58:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.795+0000 7f8441e00640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 0x7f8444151b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f843400c820 tx=0x7f843400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:16.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.796+0000 7f843a000640 1 -- 172.21.15.82:0/580963037 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434007a30 con 0x7f844406bf80 2024-03-16T17:58:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.796+0000 7f843a000640 1 -- 172.21.15.82:0/580963037 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8434007b90 con 0x7f844406bf80 2024-03-16T17:58:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.796+0000 7f843a000640 1 -- 172.21.15.82:0/580963037 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843400ce70 con 0x7f844406bf80 2024-03-16T17:58:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.796+0000 7f8448e00640 1 -- 172.21.15.82:0/580963037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84440c73d0 con 0x7f844406bf80 2024-03-16T17:58:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.796+0000 7f8448e00640 1 -- 172.21.15.82:0/580963037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84440c7990 con 0x7f844406bf80 2024-03-16T17:58:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.797+0000 7f8448e00640 1 -- 172.21.15.82:0/580963037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844406f7b0 con 0x7f844406bf80 2024-03-16T17:58:16.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.798+0000 7f843a000640 1 -- 172.21.15.82:0/580963037 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f84340113f0 con 0x7f844406bf80 2024-03-16T17:58:16.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.799+0000 7f843a000640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f842406b3b0 0x7f842406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:16.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.799+0000 7f843a000640 1 -- 172.21.15.82:0/580963037 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8434089ea0 con 0x7f844406bf80 2024-03-16T17:58:16.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.799+0000 7f8441400640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f842406b3b0 0x7f842406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:16.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.800+0000 7f8441400640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f842406b3b0 0x7f842406d870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f84440c6d50 tx=0x7f8430009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:16.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.804+0000 7f843a000640 1 -- 172.21.15.82:0/580963037 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8434054a70 con 0x7f844406bf80 2024-03-16T17:58:17.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:16.999+0000 7f8448e00640 1 -- 172.21.15.82:0/580963037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8444068640 con 0x7f844406bf80 2024-03-16T17:58:17.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.000+0000 7f843a000640 1 -- 172.21.15.82:0/580963037 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f843401f0a0 con 0x7f844406bf80 2024-03-16T17:58:17.003 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:58:17.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 -- 172.21.15.82:0/580963037 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f842406b3b0 msgr2=0x7f842406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f842406b3b0 0x7f842406d870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f84440c6d50 tx=0x7f8430009040 comp rx=0 tx=0).stop 2024-03-16T17:58:17.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 -- 172.21.15.82:0/580963037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 msgr2=0x7f8444151b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 0x7f8444151b00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f843400c820 tx=0x7f843400cc20 comp rx=0 tx=0).stop 2024-03-16T17:58:17.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 -- 172.21.15.82:0/580963037 shutdown_connections 2024-03-16T17:58:17.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f842406b3b0 0x7f842406d870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8444161b60 0x7f844415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f844406fd10 0x7f84441e8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 --2- 172.21.15.82:0/580963037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f844406bf80 0x7f8444151b00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 -- 172.21.15.82:0/580963037 >> 172.21.15.82:0/580963037 conn(0x7f84440d1bd0 msgr2=0x7f84440d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:17.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.002+0000 7f8448e00640 1 -- 172.21.15.82:0/580963037 shutdown_connections 2024-03-16T17:58:17.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.003+0000 7f8448e00640 1 -- 172.21.15.82:0/580963037 wait complete. 2024-03-16T17:58:17.024 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:58:17.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.230+0000 7f8881400640 1 Processor -- start 2024-03-16T17:58:17.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.230+0000 7f8881400640 1 -- start start 2024-03-16T17:58:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.232+0000 7f8881400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f887c0d0ba0 0x7f887c0d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.232+0000 7f8881400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0d1790 con 0x7f887c15dd40 2024-03-16T17:58:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.232+0000 7f8881400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f887c0d1a70 con 0x7f887c0b28d0 2024-03-16T17:58:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.232+0000 7f8881400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887c0cf410 con 0x7f887c0d0ba0 2024-03-16T17:58:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.232+0000 7f887be00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f887c15dd40 0x7f887c0d0490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47614/0 (socket says 172.21.15.82:47614) 2024-03-16T17:58:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.232+0000 7f887be00640 1 -- 172.21.15.82:0/1678310712 learned_addr learned my addr 172.21.15.82:0/1678310712 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.232+0000 7f887b400640 1 --2- 172.21.15.82:0/1678310712 >> v2:172.21.15.82:3300/0 conn(0x7f887c0d0ba0 0x7f887c0d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.232+0000 7f887aa00640 1 -- 172.21.15.82:0/1678310712 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402508306 0 0) 0x7f887c0d1790 con 0x7f887c15dd40 2024-03-16T17:58:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.232+0000 7f887aa00640 1 -- 172.21.15.82:0/1678310712 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8860003520 con 0x7f887c15dd40 2024-03-16T17:58:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.232+0000 7f887b400640 1 -- 172.21.15.82:0/1678310712 >> v1:172.21.15.90:6789/0 conn(0x7f887c0b28d0 legacy=0x7f887c0d1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.233+0000 7f887b400640 1 -- 172.21.15.82:0/1678310712 >> v1:172.21.15.82:6789/0 conn(0x7f887c15dd40 legacy=0x7f887c0d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.233+0000 7f887b400640 1 -- 172.21.15.82:0/1678310712 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887c0cf6b0 con 0x7f887c0d0ba0 2024-03-16T17:58:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.234+0000 7f887b400640 1 --2- 172.21.15.82:0/1678310712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0d0ba0 0x7f887c0d46e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f88680097a0 tx=0x7f886802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb0fc37303a1226 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.234+0000 7f887aa00640 1 -- 172.21.15.82:0/1678310712 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886802e070 con 0x7f887c0d0ba0 2024-03-16T17:58:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.234+0000 7f887aa00640 1 -- 172.21.15.82:0/1678310712 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88680315a0 con 0x7f887c0d0ba0 2024-03-16T17:58:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.234+0000 7f887aa00640 1 -- 172.21.15.82:0/1678310712 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8868031930 con 0x7f887c0d0ba0 2024-03-16T17:58:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.234+0000 7f8881400640 1 -- 172.21.15.82:0/1678310712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0d0ba0 msgr2=0x7f887c0d46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.234+0000 7f8881400640 1 --2- 172.21.15.82:0/1678310712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0d0ba0 0x7f887c0d46e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f88680097a0 tx=0x7f886802b840 comp rx=0 tx=0).stop 2024-03-16T17:58:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.235+0000 7f8881400640 1 -- 172.21.15.82:0/1678310712 shutdown_connections 2024-03-16T17:58:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.235+0000 7f8881400640 1 --2- 172.21.15.82:0/1678310712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0d0ba0 0x7f887c0d46e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.235+0000 7f8881400640 1 -- 172.21.15.82:0/1678310712 >> 172.21.15.82:0/1678310712 conn(0x7f887c0d3bc0 msgr2=0x7f887c0d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.235+0000 7f8881400640 1 -- 172.21.15.82:0/1678310712 shutdown_connections 2024-03-16T17:58:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.235+0000 7f8881400640 1 -- 172.21.15.82:0/1678310712 wait complete. 2024-03-16T17:58:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.236+0000 7f8881400640 1 Processor -- start 2024-03-16T17:58:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.236+0000 7f8881400640 1 -- start start 2024-03-16T17:58:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.237+0000 7f8881400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0b28d0 0x7f887c0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.237+0000 7f8881400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f887c0d0ba0 0x7f887c150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.237+0000 7f8881400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f887c15dd40 0x7f887c1f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.237+0000 7f8881400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c1616c0 con 0x7f887c0b28d0 2024-03-16T17:58:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.237+0000 7f8881400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887c0cb1d0 con 0x7f887c0d0ba0 2024-03-16T17:58:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.237+0000 7f8881400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887c15a3e0 con 0x7f887c15dd40 2024-03-16T17:58:17.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f887be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0b28d0 0x7f887c0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:17.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f887be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0b28d0 0x7f887c0cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47060/0 (socket says 172.21.15.82:47060) 2024-03-16T17:58:17.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f887be00640 1 -- 172.21.15.82:0/1202552634 learned_addr learned my addr 172.21.15.82:0/1202552634 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f887b400640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f887c0d0ba0 0x7f887c150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f887be00640 1 -- 172.21.15.82:0/1202552634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f887c15dd40 msgr2=0x7f887c1f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f887be00640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f887c15dd40 0x7f887c1f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f887be00640 1 -- 172.21.15.82:0/1202552634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f887c0d0ba0 msgr2=0x7f887c150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f887be00640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f887c0d0ba0 0x7f887c150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f887be00640 1 -- 172.21.15.82:0/1202552634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8868009400 con 0x7f887c0b28d0 2024-03-16T17:58:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f887be00640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0b28d0 0x7f887c0cac90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c390 tx=0x7f886c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.238+0000 7f8873e00640 1 -- 172.21.15.82:0/1202552634 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c005ce0 con 0x7f887c0b28d0 2024-03-16T17:58:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.239+0000 7f8873e00640 1 -- 172.21.15.82:0/1202552634 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c005e40 con 0x7f887c0b28d0 2024-03-16T17:58:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.239+0000 7f8873e00640 1 -- 172.21.15.82:0/1202552634 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0072f0 con 0x7f887c0b28d0 2024-03-16T17:58:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.239+0000 7f8881400640 1 -- 172.21.15.82:0/1202552634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f887c15e2a0 con 0x7f887c0b28d0 2024-03-16T17:58:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.239+0000 7f8881400640 1 -- 172.21.15.82:0/1202552634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f887c0cd2a0 con 0x7f887c0b28d0 2024-03-16T17:58:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.240+0000 7f8881400640 1 -- 172.21.15.82:0/1202552634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c0b20c0 con 0x7f887c0b28d0 2024-03-16T17:58:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.241+0000 7f8873e00640 1 -- 172.21.15.82:0/1202552634 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f886c007450 con 0x7f887c0b28d0 2024-03-16T17:58:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.241+0000 7f8873e00640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f885c06b3b0 0x7f885c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.242+0000 7f8873e00640 1 -- 172.21.15.82:0/1202552634 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f886c0891e0 con 0x7f887c0b28d0 2024-03-16T17:58:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.242+0000 7f887b400640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f885c06b3b0 0x7f885c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:17.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.242+0000 7f887b400640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f885c06b3b0 0x7f885c06d870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8868036000 tx=0x7f886803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:17.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.246+0000 7f8873e00640 1 -- 172.21.15.82:0/1202552634 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f886c053e40 con 0x7f887c0b28d0 2024-03-16T17:58:17.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.436+0000 7f8881400640 1 -- 172.21.15.82:0/1202552634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f887c04f250 con 0x7f887c0b28d0 2024-03-16T17:58:17.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.436+0000 7f8873e00640 1 -- 172.21.15.82:0/1202552634 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f886c057500 con 0x7f887c0b28d0 2024-03-16T17:58:17.440 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:58:17.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 -- 172.21.15.82:0/1202552634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f885c06b3b0 msgr2=0x7f885c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f885c06b3b0 0x7f885c06d870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8868036000 tx=0x7f886803d040 comp rx=0 tx=0).stop 2024-03-16T17:58:17.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 -- 172.21.15.82:0/1202552634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0b28d0 msgr2=0x7f887c0cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0b28d0 0x7f887c0cac90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f886c00c390 tx=0x7f886c00c850 comp rx=0 tx=0).stop 2024-03-16T17:58:17.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 -- 172.21.15.82:0/1202552634 shutdown_connections 2024-03-16T17:58:17.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f885c06b3b0 0x7f885c06d870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f887c15dd40 0x7f887c1f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f887c0d0ba0 0x7f887c150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 --2- 172.21.15.82:0/1202552634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f887c0b28d0 0x7f887c0cac90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 -- 172.21.15.82:0/1202552634 >> 172.21.15.82:0/1202552634 conn(0x7f887c0d3bc0 msgr2=0x7f887c0b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:17.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.439+0000 7f8881400640 1 -- 172.21.15.82:0/1202552634 shutdown_connections 2024-03-16T17:58:17.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.440+0000 7f8881400640 1 -- 172.21.15.82:0/1202552634 wait complete. 2024-03-16T17:58:17.459 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:17.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.655+0000 7fd77a200640 1 Processor -- start 2024-03-16T17:58:17.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.655+0000 7fd77a200640 1 -- start start 2024-03-16T17:58:17.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.657+0000 7fd77a200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd7741454d0 0x7fd7740b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.657+0000 7fd77a200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd7740d0040 0x7fd774070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.657+0000 7fd77a200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7740b4b90 con 0x7fd7740cdf80 2024-03-16T17:58:17.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.657+0000 7fd77a200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7740cf500 con 0x7fd7741454d0 2024-03-16T17:58:17.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.657+0000 7fd77a200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7741458b0 con 0x7fd7740d0040 2024-03-16T17:58:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.657+0000 7fd773400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd7741454d0 0x7fd7740b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.657+0000 7fd773400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd7741454d0 0x7fd7740b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47068/0 (socket says 172.21.15.82:47068) 2024-03-16T17:58:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.657+0000 7fd772a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd7740cdf80 0x7fd7740cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47626/0 (socket says 172.21.15.82:47626) 2024-03-16T17:58:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.657+0000 7fd773400640 1 -- 172.21.15.82:0/3311118881 learned_addr learned my addr 172.21.15.82:0/3311118881 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.657+0000 7fd773e00640 1 --2- 172.21.15.82:0/3311118881 >> v2:172.21.15.90:3300/0 conn(0x7fd7740d0040 0x7fd774070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.658+0000 7fd773400640 1 -- 172.21.15.82:0/3311118881 >> v2:172.21.15.90:3300/0 conn(0x7fd7740d0040 msgr2=0x7fd774070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.658+0000 7fd773400640 1 --2- 172.21.15.82:0/3311118881 >> v2:172.21.15.90:3300/0 conn(0x7fd7740d0040 0x7fd774070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.658+0000 7fd773400640 1 -- 172.21.15.82:0/3311118881 >> v1:172.21.15.82:6789/0 conn(0x7fd7740cdf80 legacy=0x7fd7740cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.658+0000 7fd773400640 1 -- 172.21.15.82:0/3311118881 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7740ce460 con 0x7fd7741454d0 2024-03-16T17:58:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.658+0000 7fd773e00640 1 --2- 172.21.15.82:0/3311118881 >> v2:172.21.15.90:3300/0 conn(0x7fd7740d0040 0x7fd774070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:58:17.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.658+0000 7fd772000640 1 -- 172.21.15.82:0/3311118881 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884877114 0 0) 0x7fd7740b4b90 con 0x7fd7740cdf80 2024-03-16T17:58:17.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.659+0000 7fd773400640 1 --2- 172.21.15.82:0/3311118881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 0x7fd7740b34c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd7740af6c0 tx=0x7fd76402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2d153cd4e71ebc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:17.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.659+0000 7fd772000640 1 -- 172.21.15.82:0/3311118881 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764030860 con 0x7fd7741454d0 2024-03-16T17:58:17.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.659+0000 7fd772000640 1 -- 172.21.15.82:0/3311118881 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7640309c0 con 0x7fd7741454d0 2024-03-16T17:58:17.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.659+0000 7fd772000640 1 -- 172.21.15.82:0/3311118881 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764030d50 con 0x7fd7741454d0 2024-03-16T17:58:17.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.660+0000 7fd77a200640 1 -- 172.21.15.82:0/3311118881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 msgr2=0x7fd7740b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.660+0000 7fd77a200640 1 --2- 172.21.15.82:0/3311118881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 0x7fd7740b34c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd7740af6c0 tx=0x7fd76402b9a0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.660+0000 7fd77a200640 1 -- 172.21.15.82:0/3311118881 shutdown_connections 2024-03-16T17:58:17.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.660+0000 7fd77a200640 1 --2- 172.21.15.82:0/3311118881 >> v2:172.21.15.90:3300/0 conn(0x7fd7740d0040 0x7fd774070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.660+0000 7fd77a200640 1 --2- 172.21.15.82:0/3311118881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 0x7fd7740b34c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.660+0000 7fd77a200640 1 -- 172.21.15.82:0/3311118881 >> 172.21.15.82:0/3311118881 conn(0x7fd77406ae70 msgr2=0x7fd77406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.660+0000 7fd77a200640 1 -- 172.21.15.82:0/3311118881 shutdown_connections 2024-03-16T17:58:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.660+0000 7fd77a200640 1 -- 172.21.15.82:0/3311118881 wait complete. 2024-03-16T17:58:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.662+0000 7fd77a200640 1 Processor -- start 2024-03-16T17:58:17.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.662+0000 7fd77a200640 1 -- start start 2024-03-16T17:58:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.663+0000 7fd77a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7740cdf80 0x7fd774157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.663+0000 7fd77a200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7740d0040 0x7fd77413aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.663+0000 7fd77a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 0x7fd77415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.663+0000 7fd77a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd774159500 con 0x7fd7741454d0 2024-03-16T17:58:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.663+0000 7fd77a200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77406c9e0 con 0x7fd7740cdf80 2024-03-16T17:58:17.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.663+0000 7fd77a200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd774158460 con 0x7fd7740d0040 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.663+0000 7fd773e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 0x7fd77415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.663+0000 7fd773e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 0x7fd77415a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47070/0 (socket says 172.21.15.82:47070) 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.663+0000 7fd773e00640 1 -- 172.21.15.82:0/699195719 learned_addr learned my addr 172.21.15.82:0/699195719 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.663+0000 7fd773400640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7740cdf80 0x7fd774157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd772a00640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7740d0040 0x7fd77413aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd773e00640 1 -- 172.21.15.82:0/699195719 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7740d0040 msgr2=0x7fd77413aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd773e00640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7740d0040 0x7fd77413aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd773e00640 1 -- 172.21.15.82:0/699195719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7740cdf80 msgr2=0x7fd774157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd773e00640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7740cdf80 0x7fd774157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd773e00640 1 -- 172.21.15.82:0/699195719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7640094d0 con 0x7fd7741454d0 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd773400640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7740cdf80 0x7fd774157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd773e00640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 0x7fd77415a000 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd76c007ef0 tx=0x7fd76c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd76b400640 1 -- 172.21.15.82:0/699195719 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c003b90 con 0x7fd7741454d0 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd76b400640 1 -- 172.21.15.82:0/699195719 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c00f040 con 0x7fd7741454d0 2024-03-16T17:58:17.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd77a200640 1 -- 172.21.15.82:0/699195719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd774158700 con 0x7fd7741454d0 2024-03-16T17:58:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.664+0000 7fd76b400640 1 -- 172.21.15.82:0/699195719 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c00de80 con 0x7fd7741454d0 2024-03-16T17:58:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.665+0000 7fd77a200640 1 -- 172.21.15.82:0/699195719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd774158cc0 con 0x7fd7741454d0 2024-03-16T17:58:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.666+0000 7fd77a200640 1 -- 172.21.15.82:0/699195719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd774067f30 con 0x7fd7741454d0 2024-03-16T17:58:17.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.666+0000 7fd76b400640 1 -- 172.21.15.82:0/699195719 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fd76c004030 con 0x7fd7741454d0 2024-03-16T17:58:17.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.667+0000 7fd76b400640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd75406b3b0 0x7fd75406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:17.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.667+0000 7fd76b400640 1 -- 172.21.15.82:0/699195719 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd76c089df0 con 0x7fd7741454d0 2024-03-16T17:58:17.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.668+0000 7fd773400640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd75406b3b0 0x7fd75406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:17.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.668+0000 7fd773400640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd75406b3b0 0x7fd75406d870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd764036000 tx=0x7fd76403b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:17.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.672+0000 7fd76b400640 1 -- 172.21.15.82:0/699195719 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd76c054b70 con 0x7fd7741454d0 2024-03-16T17:58:17.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.871+0000 7fd77a200640 1 -- 172.21.15.82:0/699195719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd77406b500 con 0x7fd7741454d0 2024-03-16T17:58:17.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.874+0000 7fd76b400640 1 -- 172.21.15.82:0/699195719 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fd76c01a090 con 0x7fd7741454d0 2024-03-16T17:58:17.878 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:17.878 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:17.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.876+0000 7fd77a200640 1 -- 172.21.15.82:0/699195719 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd75406b3b0 msgr2=0x7fd75406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.876+0000 7fd77a200640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd75406b3b0 0x7fd75406d870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd764036000 tx=0x7fd76403b040 comp rx=0 tx=0).stop 2024-03-16T17:58:17.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.876+0000 7fd77a200640 1 -- 172.21.15.82:0/699195719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 msgr2=0x7fd77415a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:17.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.876+0000 7fd77a200640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 0x7fd77415a000 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd76c007ef0 tx=0x7fd76c00d4a0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.877+0000 7fd77a200640 1 -- 172.21.15.82:0/699195719 shutdown_connections 2024-03-16T17:58:17.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.877+0000 7fd77a200640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd75406b3b0 0x7fd75406d870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.877+0000 7fd77a200640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7741454d0 0x7fd77415a000 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.877+0000 7fd77a200640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd7740d0040 0x7fd77413aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.877+0000 7fd77a200640 1 --2- 172.21.15.82:0/699195719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7740cdf80 0x7fd774157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:17.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.877+0000 7fd77a200640 1 -- 172.21.15.82:0/699195719 >> 172.21.15.82:0/699195719 conn(0x7fd77406ae70 msgr2=0x7fd77413c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:17.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.877+0000 7fd77a200640 1 -- 172.21.15.82:0/699195719 shutdown_connections 2024-03-16T17:58:17.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:17.877+0000 7fd77a200640 1 -- 172.21.15.82:0/699195719 wait complete. 2024-03-16T17:58:17.899 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:17.900 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:18.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.107+0000 7f7b3bc00640 1 Processor -- start 2024-03-16T17:58:18.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.108+0000 7f7b3bc00640 1 -- start start 2024-03-16T17:58:18.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.109+0000 7f7b3bc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7b340b56b0 0x7f7b340b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.109+0000 7f7b3bc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3406f930 0x7f7b3406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.109+0000 7f7b3bc00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3415e3a0 con 0x7f7b3406f550 2024-03-16T17:58:18.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.109+0000 7f7b3bc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3406c340 con 0x7f7b340b56b0 2024-03-16T17:58:18.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.109+0000 7f7b3bc00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3406c620 con 0x7f7b3406f930 2024-03-16T17:58:18.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.109+0000 7f7b33e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7b340b56b0 0x7f7b340b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.109+0000 7f7b33e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7b340b56b0 0x7f7b340b6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47078/0 (socket says 172.21.15.82:47078) 2024-03-16T17:58:18.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.109+0000 7f7b33e00640 1 -- 172.21.15.82:0/159900255 learned_addr learned my addr 172.21.15.82:0/159900255 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:18.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.110+0000 7f7b39c00640 1 --2- 172.21.15.82:0/159900255 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3406f930 0x7f7b3406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.110+0000 7f7b33e00640 1 -- 172.21.15.82:0/159900255 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3406f930 msgr2=0x7f7b3406d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:18.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.110+0000 7f7b33e00640 1 --2- 172.21.15.82:0/159900255 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3406f930 0x7f7b3406d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.110+0000 7f7b33e00640 1 -- 172.21.15.82:0/159900255 >> v1:172.21.15.90:6789/0 conn(0x7f7b3406f550 legacy=0x7f7b340ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:18.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.110+0000 7f7b33e00640 1 -- 172.21.15.82:0/159900255 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3406c900 con 0x7f7b340b56b0 2024-03-16T17:58:18.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.111+0000 7f7b33e00640 1 --2- 172.21.15.82:0/159900255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 0x7f7b340b6510 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009870 tx=0x7f7b1c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fa7b3edd7b51fb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.111+0000 7f7b33400640 1 -- 172.21.15.82:0/159900255 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c031640 con 0x7f7b340b56b0 2024-03-16T17:58:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.111+0000 7f7b33400640 1 -- 172.21.15.82:0/159900255 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1c0317a0 con 0x7f7b340b56b0 2024-03-16T17:58:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.111+0000 7f7b33400640 1 -- 172.21.15.82:0/159900255 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c031b30 con 0x7f7b340b56b0 2024-03-16T17:58:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.112+0000 7f7b3bc00640 1 -- 172.21.15.82:0/159900255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 msgr2=0x7f7b340b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.112+0000 7f7b3bc00640 1 --2- 172.21.15.82:0/159900255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 0x7f7b340b6510 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009870 tx=0x7f7b1c02b8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.112+0000 7f7b3bc00640 1 -- 172.21.15.82:0/159900255 shutdown_connections 2024-03-16T17:58:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.112+0000 7f7b3bc00640 1 --2- 172.21.15.82:0/159900255 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3406f930 0x7f7b3406d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.112+0000 7f7b3bc00640 1 --2- 172.21.15.82:0/159900255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 0x7f7b340b6510 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.112+0000 7f7b3bc00640 1 -- 172.21.15.82:0/159900255 >> 172.21.15.82:0/159900255 conn(0x7f7b3409a9e0 msgr2=0x7f7b3409adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.112+0000 7f7b3bc00640 1 -- 172.21.15.82:0/159900255 shutdown_connections 2024-03-16T17:58:18.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.112+0000 7f7b3bc00640 1 -- 172.21.15.82:0/159900255 wait complete. 2024-03-16T17:58:18.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.114+0000 7f7b3bc00640 1 Processor -- start 2024-03-16T17:58:18.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.114+0000 7f7b3bc00640 1 -- start start 2024-03-16T17:58:18.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.115+0000 7f7b3bc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3406f550 0x7f7b3414b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.115+0000 7f7b3bc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3406f930 0x7f7b3406d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.115+0000 7f7b3bc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 0x7f7b34152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.115+0000 7f7b3bc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3414b090 con 0x7f7b340b56b0 2024-03-16T17:58:18.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.115+0000 7f7b3bc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34157fa0 con 0x7f7b3406f930 2024-03-16T17:58:18.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.115+0000 7f7b3bc00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b34155ee0 con 0x7f7b3406f550 2024-03-16T17:58:18.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.115+0000 7f7b39c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 0x7f7b34152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.115+0000 7f7b39c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 0x7f7b34152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47084/0 (socket says 172.21.15.82:47084) 2024-03-16T17:58:18.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.115+0000 7f7b39c00640 1 -- 172.21.15.82:0/3562911758 learned_addr learned my addr 172.21.15.82:0/3562911758 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:18.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b39200640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3406f550 0x7f7b3414b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b33e00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3406f930 0x7f7b3406d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b39c00640 1 -- 172.21.15.82:0/3562911758 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3406f550 msgr2=0x7f7b3414b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b39c00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3406f550 0x7f7b3414b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b39c00640 1 -- 172.21.15.82:0/3562911758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3406f930 msgr2=0x7f7b3406d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b39c00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3406f930 0x7f7b3406d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b39c00640 1 -- 172.21.15.82:0/3562911758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0094d0 con 0x7f7b340b56b0 2024-03-16T17:58:18.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b39c00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 0x7f7b34152260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00c550 tx=0x7f7b2c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:18.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b30c00640 1 -- 172.21.15.82:0/3562911758 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c004220 con 0x7f7b340b56b0 2024-03-16T17:58:18.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b30c00640 1 -- 172.21.15.82:0/3562911758 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c004380 con 0x7f7b340b56b0 2024-03-16T17:58:18.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.116+0000 7f7b3bc00640 1 -- 172.21.15.82:0/3562911758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b341561c0 con 0x7f7b340b56b0 2024-03-16T17:58:18.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.117+0000 7f7b30c00640 1 -- 172.21.15.82:0/3562911758 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c003830 con 0x7f7b340b56b0 2024-03-16T17:58:18.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.117+0000 7f7b3bc00640 1 -- 172.21.15.82:0/3562911758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b34156780 con 0x7f7b340b56b0 2024-03-16T17:58:18.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.118+0000 7f7b3bc00640 1 -- 172.21.15.82:0/3562911758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b34156970 con 0x7f7b340b56b0 2024-03-16T17:58:18.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.119+0000 7f7b30c00640 1 -- 172.21.15.82:0/3562911758 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f7b2c003990 con 0x7f7b340b56b0 2024-03-16T17:58:18.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.119+0000 7f7b30c00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1006b3b0 0x7f7b1006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.120+0000 7f7b30c00640 1 -- 172.21.15.82:0/3562911758 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b2c089850 con 0x7f7b340b56b0 2024-03-16T17:58:18.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.120+0000 7f7b39200640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1006b3b0 0x7f7b1006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.120+0000 7f7b39200640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1006b3b0 0x7f7b1006d870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009f20 tx=0x7f7b240094d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:18.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.124+0000 7f7b30c00640 1 -- 172.21.15.82:0/3562911758 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7b2c054540 con 0x7f7b340b56b0 2024-03-16T17:58:18.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.325+0000 7f7b3bc00640 1 -- 172.21.15.82:0/3562911758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b34067f30 con 0x7f7b340b56b0 2024-03-16T17:58:18.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.326+0000 7f7b30c00640 1 -- 172.21.15.82:0/3562911758 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b2c01e020 con 0x7f7b340b56b0 2024-03-16T17:58:18.329 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:18.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.328+0000 7f7b3bc00640 1 -- 172.21.15.82:0/3562911758 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1006b3b0 msgr2=0x7f7b1006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.328+0000 7f7b3bc00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1006b3b0 0x7f7b1006d870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009f20 tx=0x7f7b240094d0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.328+0000 7f7b3bc00640 1 -- 172.21.15.82:0/3562911758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 msgr2=0x7f7b34152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.328+0000 7f7b3bc00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 0x7f7b34152260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00c550 tx=0x7f7b2c00ca10 comp rx=0 tx=0).stop 2024-03-16T17:58:18.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.329+0000 7f7b3bc00640 1 -- 172.21.15.82:0/3562911758 shutdown_connections 2024-03-16T17:58:18.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.329+0000 7f7b3bc00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1006b3b0 0x7f7b1006d870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.329+0000 7f7b3bc00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b340b56b0 0x7f7b34152260 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.329+0000 7f7b3bc00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3406f930 0x7f7b3406d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.329+0000 7f7b3bc00640 1 --2- 172.21.15.82:0/3562911758 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3406f550 0x7f7b3414b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.329+0000 7f7b3bc00640 1 -- 172.21.15.82:0/3562911758 >> 172.21.15.82:0/3562911758 conn(0x7f7b3409a9e0 msgr2=0x7f7b34158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:18.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.329+0000 7f7b3bc00640 1 -- 172.21.15.82:0/3562911758 shutdown_connections 2024-03-16T17:58:18.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.329+0000 7f7b3bc00640 1 -- 172.21.15.82:0/3562911758 wait complete. 2024-03-16T17:58:18.349 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:58:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.557+0000 7f77d2000640 1 Processor -- start 2024-03-16T17:58:18.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.557+0000 7f77d2000640 1 -- start start 2024-03-16T17:58:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.559+0000 7f77d2000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77cc1443c0 0x7f77cc06e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.559+0000 7f77d2000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f77cc0b3bc0 0x7f77cc0b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.559+0000 7f77d2000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77cc072560 con 0x7f77cc0cc450 2024-03-16T17:58:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.559+0000 7f77d2000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77cc06e850 con 0x7f77cc0b3bc0 2024-03-16T17:58:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.559+0000 7f77d2000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc070fa0 con 0x7f77cc1443c0 2024-03-16T17:58:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.559+0000 7f77cbe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f77cc0cc450 0x7f77cc070890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47630/0 (socket says 172.21.15.82:47630) 2024-03-16T17:58:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.559+0000 7f77cbe00640 1 -- 172.21.15.82:0/601503317 learned_addr learned my addr 172.21.15.82:0/601503317 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.559+0000 7f77d1600640 1 --2- 172.21.15.82:0/601503317 >> v2:172.21.15.82:3300/0 conn(0x7f77cc0b3bc0 0x7f77cc0b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.559+0000 7f77d0c00640 1 --2- 172.21.15.82:0/601503317 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77cc1443c0 0x7f77cc06e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.559+0000 7f77cb400640 1 -- 172.21.15.82:0/601503317 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788755534 0 0) 0x7f77cc072560 con 0x7f77cc0cc450 2024-03-16T17:58:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.560+0000 7f77cb400640 1 -- 172.21.15.82:0/601503317 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77b0003520 con 0x7f77cc0cc450 2024-03-16T17:58:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.560+0000 7f77d1600640 1 -- 172.21.15.82:0/601503317 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77cc1443c0 msgr2=0x7f77cc06e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.560+0000 7f77d1600640 1 --2- 172.21.15.82:0/601503317 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77cc1443c0 0x7f77cc06e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.560+0000 7f77d1600640 1 -- 172.21.15.82:0/601503317 >> v1:172.21.15.82:6789/0 conn(0x7f77cc0cc450 legacy=0x7f77cc070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.560+0000 7f77d1600640 1 -- 172.21.15.82:0/601503317 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77cc1447a0 con 0x7f77cc0b3bc0 2024-03-16T17:58:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.560+0000 7f77d0c00640 1 --2- 172.21.15.82:0/601503317 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77cc1443c0 0x7f77cc06e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:58:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.561+0000 7f77d1600640 1 --2- 172.21.15.82:0/601503317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0b3bc0 0x7f77cc0b4030 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f77c40097a0 tx=0x7f77c402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e9ff863f697cb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.561+0000 7f77cb400640 1 -- 172.21.15.82:0/601503317 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c402d040 con 0x7f77cc0b3bc0 2024-03-16T17:58:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.561+0000 7f77cb400640 1 -- 172.21.15.82:0/601503317 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c4031810 con 0x7f77cc0b3bc0 2024-03-16T17:58:18.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.561+0000 7f77cb400640 1 -- 172.21.15.82:0/601503317 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4031ba0 con 0x7f77cc0b3bc0 2024-03-16T17:58:18.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.562+0000 7f77d2000640 1 -- 172.21.15.82:0/601503317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0b3bc0 msgr2=0x7f77cc0b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.562+0000 7f77d2000640 1 --2- 172.21.15.82:0/601503317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0b3bc0 0x7f77cc0b4030 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f77c40097a0 tx=0x7f77c402b8b0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.562+0000 7f77d2000640 1 -- 172.21.15.82:0/601503317 shutdown_connections 2024-03-16T17:58:18.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.562+0000 7f77d2000640 1 --2- 172.21.15.82:0/601503317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0b3bc0 0x7f77cc0b4030 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.562+0000 7f77d2000640 1 --2- 172.21.15.82:0/601503317 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77cc1443c0 0x7f77cc06e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.562+0000 7f77d2000640 1 -- 172.21.15.82:0/601503317 >> 172.21.15.82:0/601503317 conn(0x7f77cc06ae70 msgr2=0x7f77cc06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:18.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.562+0000 7f77d2000640 1 -- 172.21.15.82:0/601503317 shutdown_connections 2024-03-16T17:58:18.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.562+0000 7f77d2000640 1 -- 172.21.15.82:0/601503317 wait complete. 2024-03-16T17:58:18.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.564+0000 7f77d2000640 1 Processor -- start 2024-03-16T17:58:18.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.564+0000 7f77d2000640 1 -- start start 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.564+0000 7f77d2000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77cc0b3bc0 0x7f77cc15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77d2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0cc450 0x7f77cc13eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77d2000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77cc1443c0 0x7f77cc13f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77d2000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc15d0a0 con 0x7f77cc0cc450 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77d2000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc1f00f0 con 0x7f77cc0b3bc0 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77d2000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77cc15d340 con 0x7f77cc1443c0 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77cbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0cc450 0x7f77cc13eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77cbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0cc450 0x7f77cc13eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47100/0 (socket says 172.21.15.82:47100) 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77cbe00640 1 -- 172.21.15.82:0/2382706724 learned_addr learned my addr 172.21.15.82:0/2382706724 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77cbe00640 1 -- 172.21.15.82:0/2382706724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77cc1443c0 msgr2=0x7f77cc13f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77cbe00640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77cc1443c0 0x7f77cc13f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77d0c00640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77cc0b3bc0 0x7f77cc15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77cbe00640 1 -- 172.21.15.82:0/2382706724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77cc0b3bc0 msgr2=0x7f77cc15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77cbe00640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77cc0b3bc0 0x7f77cc15a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.565+0000 7f77cbe00640 1 -- 172.21.15.82:0/2382706724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c4009400 con 0x7f77cc0cc450 2024-03-16T17:58:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.566+0000 7f77cbe00640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0cc450 0x7f77cc13eb90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f77b400c430 tx=0x7f77b400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.566+0000 7f77c8c00640 1 -- 172.21.15.82:0/2382706724 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b4005ce0 con 0x7f77cc0cc450 2024-03-16T17:58:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.566+0000 7f77c8c00640 1 -- 172.21.15.82:0/2382706724 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b4005e40 con 0x7f77cc0cc450 2024-03-16T17:58:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.566+0000 7f77d2000640 1 -- 172.21.15.82:0/2382706724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77cc15d620 con 0x7f77cc0cc450 2024-03-16T17:58:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.566+0000 7f77c8c00640 1 -- 172.21.15.82:0/2382706724 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40072f0 con 0x7f77cc0cc450 2024-03-16T17:58:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.566+0000 7f77d2000640 1 -- 172.21.15.82:0/2382706724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77cc15de20 con 0x7f77cc0cc450 2024-03-16T17:58:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.567+0000 7f77d2000640 1 -- 172.21.15.82:0/2382706724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77cc15db40 con 0x7f77cc0cc450 2024-03-16T17:58:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.568+0000 7f77c8c00640 1 -- 172.21.15.82:0/2382706724 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f77b4007450 con 0x7f77cc0cc450 2024-03-16T17:58:18.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.569+0000 7f77c8c00640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f77a806b3b0 0x7f77a806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:18.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.569+0000 7f77c8c00640 1 -- 172.21.15.82:0/2382706724 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77b4089310 con 0x7f77cc0cc450 2024-03-16T17:58:18.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.569+0000 7f77d0c00640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f77a806b3b0 0x7f77a806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:18.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.570+0000 7f77d0c00640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f77a806b3b0 0x7f77a806d870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77cc15ce90 tx=0x7f77bc00b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:18.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.573+0000 7f77c8c00640 1 -- 172.21.15.82:0/2382706724 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f77cc15db40 con 0x7f77cc0cc450 2024-03-16T17:58:18.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.766+0000 7f77d2000640 1 -- 172.21.15.82:0/2382706724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f77cc15db40 con 0x7f77cc0cc450 2024-03-16T17:58:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.767+0000 7f77c8c00640 1 -- 172.21.15.82:0/2382706724 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f77cc15db40 con 0x7f77cc0cc450 2024-03-16T17:58:18.771 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:58:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 -- 172.21.15.82:0/2382706724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f77a806b3b0 msgr2=0x7f77a806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f77a806b3b0 0x7f77a806d870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f77cc15ce90 tx=0x7f77bc00b040 comp rx=0 tx=0).stop 2024-03-16T17:58:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 -- 172.21.15.82:0/2382706724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0cc450 msgr2=0x7f77cc13eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0cc450 0x7f77cc13eb90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f77b400c430 tx=0x7f77b400c8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 -- 172.21.15.82:0/2382706724 shutdown_connections 2024-03-16T17:58:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f77a806b3b0 0x7f77a806d870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f77cc1443c0 0x7f77cc13f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77cc0cc450 0x7f77cc13eb90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 --2- 172.21.15.82:0/2382706724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f77cc0b3bc0 0x7f77cc15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 -- 172.21.15.82:0/2382706724 >> 172.21.15.82:0/2382706724 conn(0x7f77cc06ae70 msgr2=0x7f77cc070890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:18.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 -- 172.21.15.82:0/2382706724 shutdown_connections 2024-03-16T17:58:18.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:18.770+0000 7f77d2000640 1 -- 172.21.15.82:0/2382706724 wait complete. 2024-03-16T17:58:18.792 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:58:19.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.004+0000 7f22b2600640 1 Processor -- start 2024-03-16T17:58:19.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.004+0000 7f22b2600640 1 -- start start 2024-03-16T17:58:19.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.006+0000 7f22b2600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f22ac0d2ab0 0x7f22ac15e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.006+0000 7f22b2600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f22ac0d48e0 0x7f22ac0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.006+0000 7f22b2600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ac0b33c0 con 0x7f22ac15e040 2024-03-16T17:58:19.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.006+0000 7f22b2600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac0d1cd0 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.006+0000 7f22b2600640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ac0d1fb0 con 0x7f22ac0d48e0 2024-03-16T17:58:19.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.006+0000 7f22a3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f22ac0d2ab0 0x7f22ac15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.006+0000 7f22ab400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f22ac15e040 0x7f22ac0d23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47638/0 (socket says 172.21.15.82:47638) 2024-03-16T17:58:19.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.006+0000 7f22ab400640 1 -- 172.21.15.82:0/2155630772 learned_addr learned my addr 172.21.15.82:0/2155630772 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:19.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.007+0000 7f22abe00640 1 --2- 172.21.15.82:0/2155630772 >> v2:172.21.15.90:3300/0 conn(0x7f22ac0d48e0 0x7f22ac0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.007+0000 7f22a3e00640 1 -- 172.21.15.82:0/2155630772 >> v2:172.21.15.90:3300/0 conn(0x7f22ac0d48e0 msgr2=0x7f22ac0ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:19.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.007+0000 7f22a3e00640 1 --2- 172.21.15.82:0/2155630772 >> v2:172.21.15.90:3300/0 conn(0x7f22ac0d48e0 0x7f22ac0ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.007+0000 7f22a3e00640 1 -- 172.21.15.82:0/2155630772 >> v1:172.21.15.82:6789/0 conn(0x7f22ac15e040 legacy=0x7f22ac0d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.007+0000 7f22a3e00640 1 -- 172.21.15.82:0/2155630772 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac0b3610 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.007+0000 7f22aaa00640 1 -- 172.21.15.82:0/2155630772 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036517799 0 0) 0x7f22ac0b33c0 con 0x7f22ac15e040 2024-03-16T17:58:19.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.008+0000 7f22a3e00640 1 --2- 172.21.15.82:0/2155630772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 0x7f22ac15e420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0b9e70 tx=0x7f229402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c47449e84b8049 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:19.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.009+0000 7f22aaa00640 1 -- 172.21.15.82:0/2155630772 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229402e070 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.009+0000 7f22aaa00640 1 -- 172.21.15.82:0/2155630772 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22940315a0 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.009+0000 7f22aaa00640 1 -- 172.21.15.82:0/2155630772 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294031930 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.009+0000 7f22b2600640 1 -- 172.21.15.82:0/2155630772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 msgr2=0x7f22ac15e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.009+0000 7f22b2600640 1 --2- 172.21.15.82:0/2155630772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 0x7f22ac15e420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0b9e70 tx=0x7f229402b840 comp rx=0 tx=0).stop 2024-03-16T17:58:19.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.009+0000 7f22b2600640 1 -- 172.21.15.82:0/2155630772 shutdown_connections 2024-03-16T17:58:19.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.009+0000 7f22b2600640 1 --2- 172.21.15.82:0/2155630772 >> v2:172.21.15.90:3300/0 conn(0x7f22ac0d48e0 0x7f22ac0ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.009+0000 7f22b2600640 1 --2- 172.21.15.82:0/2155630772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 0x7f22ac15e420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.009+0000 7f22b2600640 1 -- 172.21.15.82:0/2155630772 >> 172.21.15.82:0/2155630772 conn(0x7f22ac0d3dc0 msgr2=0x7f22ac0d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:19.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.010+0000 7f22b2600640 1 -- 172.21.15.82:0/2155630772 shutdown_connections 2024-03-16T17:58:19.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.010+0000 7f22b2600640 1 -- 172.21.15.82:0/2155630772 wait complete. 2024-03-16T17:58:19.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.011+0000 7f22b2600640 1 Processor -- start 2024-03-16T17:58:19.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.011+0000 7f22b2600640 1 -- start start 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.012+0000 7f22b2600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 0x7f22ac1e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.012+0000 7f22b2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22ac0d48e0 0x7f22ac150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.012+0000 7f22b2600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f22ac15e040 0x7f22ac0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.012+0000 7f22b2600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac15b9c0 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.012+0000 7f22b2600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1e5e50 con 0x7f22ac0d48e0 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.012+0000 7f22b2600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac1e6130 con 0x7f22ac15e040 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22ab400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 0x7f22ac1e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22ab400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 0x7f22ac1e57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47108/0 (socket says 172.21.15.82:47108) 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22ab400640 1 -- 172.21.15.82:0/3648423681 learned_addr learned my addr 172.21.15.82:0/3648423681 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22a3e00640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22ac0d48e0 0x7f22ac150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22ab400640 1 -- 172.21.15.82:0/3648423681 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f22ac15e040 msgr2=0x7f22ac0cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22ab400640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f22ac15e040 0x7f22ac0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22ab400640 1 -- 172.21.15.82:0/3648423681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22ac0d48e0 msgr2=0x7f22ac150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22ab400640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22ac0d48e0 0x7f22ac150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22ab400640 1 -- 172.21.15.82:0/3648423681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2294009400 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22a3e00640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22ac0d48e0 0x7f22ac150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:19.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.013+0000 7f22ab400640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 0x7f22ac1e57c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c430 tx=0x7f229c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:19.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.014+0000 7f22a3400640 1 -- 172.21.15.82:0/3648423681 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0074e0 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.014+0000 7f22a3400640 1 -- 172.21.15.82:0/3648423681 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c007660 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.014+0000 7f22a3400640 1 -- 172.21.15.82:0/3648423681 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c004a20 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.014+0000 7f22b2600640 1 -- 172.21.15.82:0/3648423681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac1e6410 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.014+0000 7f22b2600640 1 -- 172.21.15.82:0/3648423681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac1e69d0 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.015+0000 7f22b2600640 1 -- 172.21.15.82:0/3648423681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac1e5570 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.016+0000 7f22a3400640 1 -- 172.21.15.82:0/3648423681 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f229c004b80 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.017+0000 7f22a3400640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f228806b3b0 0x7f228806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.017+0000 7f22a3400640 1 -- 172.21.15.82:0/3648423681 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f229c089a30 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.017+0000 7f22a3e00640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f228806b3b0 0x7f228806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.018+0000 7f22a3e00640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f228806b3b0 0x7f228806d870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2294036000 tx=0x7f229403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:19.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.021+0000 7f22a3400640 1 -- 172.21.15.82:0/3648423681 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f229c054690 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.222+0000 7f22b2600640 1 -- 172.21.15.82:0/3648423681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f22ac067f30 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.223+0000 7f22a3400640 1 -- 172.21.15.82:0/3648423681 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f229c057d50 con 0x7f22ac0d2ab0 2024-03-16T17:58:19.226 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:58:19.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 -- 172.21.15.82:0/3648423681 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f228806b3b0 msgr2=0x7f228806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f228806b3b0 0x7f228806d870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2294036000 tx=0x7f229403d040 comp rx=0 tx=0).stop 2024-03-16T17:58:19.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 -- 172.21.15.82:0/3648423681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 msgr2=0x7f22ac1e57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 0x7f22ac1e57c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c430 tx=0x7f229c00c8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 -- 172.21.15.82:0/3648423681 shutdown_connections 2024-03-16T17:58:19.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f228806b3b0 0x7f228806d870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f22ac15e040 0x7f22ac0cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f22ac0d48e0 0x7f22ac150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 --2- 172.21.15.82:0/3648423681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22ac0d2ab0 0x7f22ac1e57c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 -- 172.21.15.82:0/3648423681 >> 172.21.15.82:0/3648423681 conn(0x7f22ac0d3dc0 msgr2=0x7f22ac0b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:19.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 -- 172.21.15.82:0/3648423681 shutdown_connections 2024-03-16T17:58:19.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.225+0000 7f22b2600640 1 -- 172.21.15.82:0/3648423681 wait complete. 2024-03-16T17:58:19.270 INFO:tasks.cephfs_test_runner:Starting test: test_acls (tasks.cephfs.test_acls.TestACLs) 2024-03-16T17:58:19.271 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:19.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.475+0000 7f5889a00640 1 Processor -- start 2024-03-16T17:58:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.475+0000 7f5889a00640 1 -- start start 2024-03-16T17:58:19.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.477+0000 7f5889a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5884144400 0x7f58840cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.477+0000 7f5889a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 0x7f58840cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.477+0000 7f5889a00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588415e820 con 0x7f588415b310 2024-03-16T17:58:19.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.477+0000 7f5889a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588415f130 con 0x7f5884144400 2024-03-16T17:58:19.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.477+0000 7f5889a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588415be90 con 0x7f588415f550 2024-03-16T17:58:19.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.477+0000 7f5882a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5884144400 0x7f58840cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.477+0000 7f5882a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5884144400 0x7f58840cf990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:38254/0 (socket says 172.21.15.82:38254) 2024-03-16T17:58:19.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.477+0000 7f5882000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 0x7f58840cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.477+0000 7f5882a00640 1 -- 172.21.15.82:0/486420374 learned_addr learned my addr 172.21.15.82:0/486420374 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:19.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.478+0000 7f5882000640 1 -- 172.21.15.82:0/486420374 >> v2:172.21.15.90:3300/0 conn(0x7f5884144400 msgr2=0x7f58840cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:19.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.478+0000 7f5882000640 1 --2- 172.21.15.82:0/486420374 >> v2:172.21.15.90:3300/0 conn(0x7f5884144400 0x7f58840cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.478+0000 7f5882000640 1 -- 172.21.15.82:0/486420374 >> v1:172.21.15.90:6789/0 conn(0x7f588415b310 legacy=0x7f588415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.478+0000 7f5882000640 1 -- 172.21.15.82:0/486420374 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588415c080 con 0x7f588415f550 2024-03-16T17:58:19.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.478+0000 7f5882a00640 1 --2- 172.21.15.82:0/486420374 >> v2:172.21.15.90:3300/0 conn(0x7f5884144400 0x7f58840cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:58:19.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.480+0000 7f5882000640 1 --2- 172.21.15.82:0/486420374 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 0x7f58840cfed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f58700097a0 tx=0x7f587002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e9225291f44621a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:19.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.480+0000 7f5881600640 1 -- 172.21.15.82:0/486420374 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870031780 con 0x7f588415f550 2024-03-16T17:58:19.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.480+0000 7f5881600640 1 -- 172.21.15.82:0/486420374 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58700318e0 con 0x7f588415f550 2024-03-16T17:58:19.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.480+0000 7f5881600640 1 -- 172.21.15.82:0/486420374 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870031c70 con 0x7f588415f550 2024-03-16T17:58:19.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.481+0000 7f5889a00640 1 -- 172.21.15.82:0/486420374 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 msgr2=0x7f58840cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.481+0000 7f5889a00640 1 --2- 172.21.15.82:0/486420374 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 0x7f58840cfed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f58700097a0 tx=0x7f587002b820 comp rx=0 tx=0).stop 2024-03-16T17:58:19.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.481+0000 7f5889a00640 1 -- 172.21.15.82:0/486420374 shutdown_connections 2024-03-16T17:58:19.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.481+0000 7f5889a00640 1 --2- 172.21.15.82:0/486420374 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 0x7f58840cfed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.481+0000 7f5889a00640 1 --2- 172.21.15.82:0/486420374 >> v2:172.21.15.90:3300/0 conn(0x7f5884144400 0x7f58840cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.481+0000 7f5889a00640 1 -- 172.21.15.82:0/486420374 >> 172.21.15.82:0/486420374 conn(0x7f588406ae70 msgr2=0x7f588406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:19.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.481+0000 7f5889a00640 1 -- 172.21.15.82:0/486420374 shutdown_connections 2024-03-16T17:58:19.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.481+0000 7f5889a00640 1 -- 172.21.15.82:0/486420374 wait complete. 2024-03-16T17:58:19.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.483+0000 7f5889a00640 1 Processor -- start 2024-03-16T17:58:19.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.483+0000 7f5889a00640 1 -- start start 2024-03-16T17:58:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.484+0000 7f5889a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5884144400 0x7f5884133da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.484+0000 7f5889a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588415b310 0x7f58841e4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.484+0000 7f5889a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 0x7f58841e4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.484+0000 7f5889a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5884132d00 con 0x7f588415b310 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.484+0000 7f5889a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5884132fe0 con 0x7f5884144400 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.484+0000 7f5889a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58841332c0 con 0x7f588415f550 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.484+0000 7f5882000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588415b310 0x7f58841e4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.485+0000 7f5882000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588415b310 0x7f58841e4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47118/0 (socket says 172.21.15.82:47118) 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.485+0000 7f5882000640 1 -- 172.21.15.82:0/1900236625 learned_addr learned my addr 172.21.15.82:0/1900236625 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.485+0000 7f5882a00640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5884144400 0x7f5884133da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.485+0000 7f5883400640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 0x7f58841e4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.485+0000 7f5882000640 1 -- 172.21.15.82:0/1900236625 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 msgr2=0x7f58841e4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.485+0000 7f5882000640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 0x7f58841e4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.485+0000 7f5882000640 1 -- 172.21.15.82:0/1900236625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5884144400 msgr2=0x7f5884133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.485+0000 7f5882000640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5884144400 0x7f5884133da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.485+0000 7f5882000640 1 -- 172.21.15.82:0/1900236625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587400b770 con 0x7f588415b310 2024-03-16T17:58:19.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.485+0000 7f5882000640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588415b310 0x7f58841e4640 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f58700098c0 tx=0x7f58700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.486+0000 7f587aa00640 1 -- 172.21.15.82:0/1900236625 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870032820 con 0x7f588415b310 2024-03-16T17:58:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.486+0000 7f587aa00640 1 -- 172.21.15.82:0/1900236625 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58700329a0 con 0x7f588415b310 2024-03-16T17:58:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.486+0000 7f5889a00640 1 -- 172.21.15.82:0/1900236625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5870009400 con 0x7f588415b310 2024-03-16T17:58:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.486+0000 7f587aa00640 1 -- 172.21.15.82:0/1900236625 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870032d50 con 0x7f588415b310 2024-03-16T17:58:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.486+0000 7f5889a00640 1 -- 172.21.15.82:0/1900236625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5884133880 con 0x7f588415b310 2024-03-16T17:58:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.487+0000 7f5889a00640 1 -- 172.21.15.82:0/1900236625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5884067f30 con 0x7f588415b310 2024-03-16T17:58:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.488+0000 7f587aa00640 1 -- 172.21.15.82:0/1900236625 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f587004d3f0 con 0x7f588415b310 2024-03-16T17:58:19.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.489+0000 7f587aa00640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f586406b3b0 0x7f586406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.489+0000 7f587aa00640 1 -- 172.21.15.82:0/1900236625 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58700ab960 con 0x7f588415b310 2024-03-16T17:58:19.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.489+0000 7f5882a00640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f586406b3b0 0x7f586406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.490+0000 7f5882a00640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f586406b3b0 0x7f586406d870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f58840b2670 tx=0x7f587400b450 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:19.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.493+0000 7f587aa00640 1 -- 172.21.15.82:0/1900236625 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5870076550 con 0x7f588415b310 2024-03-16T17:58:19.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.687+0000 7f5889a00640 1 -- 172.21.15.82:0/1900236625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f588406d460 con 0x7f588415b310 2024-03-16T17:58:19.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.689+0000 7f587aa00640 1 -- 172.21.15.82:0/1900236625 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f5870079c10 con 0x7f588415b310 2024-03-16T17:58:19.693 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:19.694 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:19.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 -- 172.21.15.82:0/1900236625 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f586406b3b0 msgr2=0x7f586406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f586406b3b0 0x7f586406d870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f58840b2670 tx=0x7f587400b450 comp rx=0 tx=0).stop 2024-03-16T17:58:19.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 -- 172.21.15.82:0/1900236625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588415b310 msgr2=0x7f58841e4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588415b310 0x7f58841e4640 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f58700098c0 tx=0x7f58700046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 -- 172.21.15.82:0/1900236625 shutdown_connections 2024-03-16T17:58:19.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f586406b3b0 0x7f586406d870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f588415f550 0x7f58841e4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f588415b310 0x7f58841e4640 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 --2- 172.21.15.82:0/1900236625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5884144400 0x7f5884133da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 -- 172.21.15.82:0/1900236625 >> 172.21.15.82:0/1900236625 conn(0x7f588406ae70 msgr2=0x7f5884070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:19.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 -- 172.21.15.82:0/1900236625 shutdown_connections 2024-03-16T17:58:19.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.692+0000 7f5889a00640 1 -- 172.21.15.82:0/1900236625 wait complete. 2024-03-16T17:58:19.713 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:19.714 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:19.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.918+0000 7fe106800640 1 Processor -- start 2024-03-16T17:58:19.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.918+0000 7fe106800640 1 -- start start 2024-03-16T17:58:19.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe106800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe100159960 0x7fe10006e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe106800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe10006e770 0x7fe1000cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe106800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1000ce550 con 0x7fe1000cc850 2024-03-16T17:58:19.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe106800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe100159d40 con 0x7fe10006e770 2024-03-16T17:58:19.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe106800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10006ec20 con 0x7fe100159960 2024-03-16T17:58:19.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe0fea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe10006e770 0x7fe1000cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe0fea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe10006e770 0x7fe1000cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47134/0 (socket says 172.21.15.82:47134) 2024-03-16T17:58:19.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe0fea00640 1 -- 172.21.15.82:0/923039237 learned_addr learned my addr 172.21.15.82:0/923039237 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:19.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe0ff400640 1 --2- 172.21.15.82:0/923039237 >> v2:172.21.15.90:3300/0 conn(0x7fe100159960 0x7fe10006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe0fea00640 1 -- 172.21.15.82:0/923039237 >> v2:172.21.15.90:3300/0 conn(0x7fe100159960 msgr2=0x7fe10006e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:19.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe0fea00640 1 --2- 172.21.15.82:0/923039237 >> v2:172.21.15.90:3300/0 conn(0x7fe100159960 0x7fe10006e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe0fea00640 1 -- 172.21.15.82:0/923039237 >> v1:172.21.15.90:6789/0 conn(0x7fe1000cc850 legacy=0x7fe10015f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:19.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.920+0000 7fe0fea00640 1 -- 172.21.15.82:0/923039237 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10015c370 con 0x7fe10006e770 2024-03-16T17:58:19.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.921+0000 7fe0fea00640 1 --2- 172.21.15.82:0/923039237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 0x7fe1000cc310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1000cc260 tx=0x7fe0ec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845867f77ec40757 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:19.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.922+0000 7fe0fe000640 1 -- 172.21.15.82:0/923039237 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0317e0 con 0x7fe10006e770 2024-03-16T17:58:19.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.922+0000 7fe0fe000640 1 -- 172.21.15.82:0/923039237 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec031940 con 0x7fe10006e770 2024-03-16T17:58:19.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.922+0000 7fe0fe000640 1 -- 172.21.15.82:0/923039237 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec031cd0 con 0x7fe10006e770 2024-03-16T17:58:19.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.922+0000 7fe106800640 1 -- 172.21.15.82:0/923039237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 msgr2=0x7fe1000cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.922+0000 7fe106800640 1 --2- 172.21.15.82:0/923039237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 0x7fe1000cc310 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1000cc260 tx=0x7fe0ec02b880 comp rx=0 tx=0).stop 2024-03-16T17:58:19.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.923+0000 7fe106800640 1 -- 172.21.15.82:0/923039237 shutdown_connections 2024-03-16T17:58:19.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.923+0000 7fe106800640 1 --2- 172.21.15.82:0/923039237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 0x7fe1000cc310 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.923+0000 7fe106800640 1 --2- 172.21.15.82:0/923039237 >> v2:172.21.15.90:3300/0 conn(0x7fe100159960 0x7fe10006e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.923+0000 7fe106800640 1 -- 172.21.15.82:0/923039237 >> 172.21.15.82:0/923039237 conn(0x7fe1000b1640 msgr2=0x7fe1000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:19.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.923+0000 7fe106800640 1 -- 172.21.15.82:0/923039237 shutdown_connections 2024-03-16T17:58:19.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.923+0000 7fe106800640 1 -- 172.21.15.82:0/923039237 wait complete. 2024-03-16T17:58:19.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.924+0000 7fe106800640 1 Processor -- start 2024-03-16T17:58:19.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.924+0000 7fe106800640 1 -- start start 2024-03-16T17:58:19.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.925+0000 7fe106800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 0x7fe10015e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.925+0000 7fe106800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1000cc850 0x7fe100142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.925+0000 7fe106800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe100159960 0x7fe100140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.925+0000 7fe106800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10015cff0 con 0x7fe10006e770 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.925+0000 7fe106800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000b4150 con 0x7fe100159960 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.925+0000 7fe106800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000b4710 con 0x7fe1000cc850 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ff400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 0x7fe10015e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ff400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 0x7fe10015e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47140/0 (socket says 172.21.15.82:47140) 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ff400640 1 -- 172.21.15.82:0/3351861184 learned_addr learned my addr 172.21.15.82:0/3351861184 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0fea00640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1000cc850 0x7fe100142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ffe00640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe100159960 0x7fe100140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ff400640 1 -- 172.21.15.82:0/3351861184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1000cc850 msgr2=0x7fe100142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ff400640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1000cc850 0x7fe100142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ff400640 1 -- 172.21.15.82:0/3351861184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe100159960 msgr2=0x7fe100140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ff400640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe100159960 0x7fe100140aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ff400640 1 -- 172.21.15.82:0/3351861184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec0094d0 con 0x7fe10006e770 2024-03-16T17:58:19.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ffe00640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe100159960 0x7fe100140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:19.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0fea00640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1000cc850 0x7fe100142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:19.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.926+0000 7fe0ff400640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 0x7fe10015e6a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000d620 tx=0x7fe0f000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:19.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.927+0000 7fe0f7400640 1 -- 172.21.15.82:0/3351861184 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000b620 con 0x7fe10006e770 2024-03-16T17:58:19.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.927+0000 7fe0f7400640 1 -- 172.21.15.82:0/3351861184 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f000b780 con 0x7fe10006e770 2024-03-16T17:58:19.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.927+0000 7fe0f7400640 1 -- 172.21.15.82:0/3351861184 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f000bb30 con 0x7fe10006e770 2024-03-16T17:58:19.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.927+0000 7fe106800640 1 -- 172.21.15.82:0/3351861184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1000a9800 con 0x7fe10006e770 2024-03-16T17:58:19.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.927+0000 7fe106800640 1 -- 172.21.15.82:0/3351861184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1000b43a0 con 0x7fe10006e770 2024-03-16T17:58:19.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.928+0000 7fe106800640 1 -- 172.21.15.82:0/3351861184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe100067f30 con 0x7fe10006e770 2024-03-16T17:58:19.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.929+0000 7fe0f7400640 1 -- 172.21.15.82:0/3351861184 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe0f000bc90 con 0x7fe10006e770 2024-03-16T17:58:19.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.930+0000 7fe0f7400640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b3b0 0x7fe0e006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:19.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.930+0000 7fe0f7400640 1 -- 172.21.15.82:0/3351861184 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0f008a1c0 con 0x7fe10006e770 2024-03-16T17:58:19.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.931+0000 7fe0fea00640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b3b0 0x7fe0e006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:19.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.931+0000 7fe0fea00640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b3b0 0x7fe0e006d870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec036000 tx=0x7fe0ec03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:19.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:19.935+0000 7fe0f7400640 1 -- 172.21.15.82:0/3351861184 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe0f0054d80 con 0x7fe10006e770 2024-03-16T17:58:20.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.133+0000 7fe106800640 1 -- 172.21.15.82:0/3351861184 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1000ceb80 con 0x7fe10006e770 2024-03-16T17:58:20.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.134+0000 7fe0f7400640 1 -- 172.21.15.82:0/3351861184 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0f0058440 con 0x7fe10006e770 2024-03-16T17:58:20.138 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:20.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.136+0000 7fe106800640 1 -- 172.21.15.82:0/3351861184 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b3b0 msgr2=0x7fe0e006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b3b0 0x7fe0e006d870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec036000 tx=0x7fe0ec03d040 comp rx=0 tx=0).stop 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 -- 172.21.15.82:0/3351861184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 msgr2=0x7fe10015e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 0x7fe10015e6a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000d620 tx=0x7fe0f000dae0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 -- 172.21.15.82:0/3351861184 shutdown_connections 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b3b0 0x7fe0e006d870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe100159960 0x7fe100140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1000cc850 0x7fe100142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 --2- 172.21.15.82:0/3351861184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe10006e770 0x7fe10015e6a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 -- 172.21.15.82:0/3351861184 >> 172.21.15.82:0/3351861184 conn(0x7fe1000b1640 msgr2=0x7fe1000afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 -- 172.21.15.82:0/3351861184 shutdown_connections 2024-03-16T17:58:20.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.137+0000 7fe106800640 1 -- 172.21.15.82:0/3351861184 wait complete. 2024-03-16T17:58:20.157 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:58:20.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.364+0000 7f2bf3800640 1 Processor -- start 2024-03-16T17:58:20.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.365+0000 7f2bf3800640 1 -- start start 2024-03-16T17:58:20.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.366+0000 7f2bf3800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2bec15e0d0 0x7f2bec0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.366+0000 7f2bf3800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2bec0d1c00 0x7f2bec0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.366+0000 7f2bf3800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec0cfc80 con 0x7f2bec0d28c0 2024-03-16T17:58:20.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.366+0000 7f2bf3800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec0d2d30 con 0x7f2bec15e0d0 2024-03-16T17:58:20.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.366+0000 7f2bf3800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec0cf9a0 con 0x7f2bec0d1c00 2024-03-16T17:58:20.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.367+0000 7f2bf0e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2bec15e0d0 0x7f2bec0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.367+0000 7f2bf0e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2bec15e0d0 0x7f2bec0cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47154/0 (socket says 172.21.15.82:47154) 2024-03-16T17:58:20.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.367+0000 7f2bf0e00640 1 -- 172.21.15.82:0/1911324428 learned_addr learned my addr 172.21.15.82:0/1911324428 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:20.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.367+0000 7f2bebe00640 1 --2- 172.21.15.82:0/1911324428 >> v2:172.21.15.90:3300/0 conn(0x7f2bec0d1c00 0x7f2bec0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.367+0000 7f2bf0e00640 1 -- 172.21.15.82:0/1911324428 >> v2:172.21.15.90:3300/0 conn(0x7f2bec0d1c00 msgr2=0x7f2bec0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:20.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.367+0000 7f2bf0e00640 1 --2- 172.21.15.82:0/1911324428 >> v2:172.21.15.90:3300/0 conn(0x7f2bec0d1c00 0x7f2bec0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.367+0000 7f2bf0e00640 1 -- 172.21.15.82:0/1911324428 >> v1:172.21.15.90:6789/0 conn(0x7f2bec0d28c0 legacy=0x7f2bec0d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:20.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.367+0000 7f2bf0e00640 1 -- 172.21.15.82:0/1911324428 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec0b37c0 con 0x7f2bec15e0d0 2024-03-16T17:58:20.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.367+0000 7f2bebe00640 1 --2- 172.21.15.82:0/1911324428 >> v2:172.21.15.90:3300/0 conn(0x7f2bec0d1c00 0x7f2bec0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:20.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.368+0000 7f2bf0e00640 1 --2- 172.21.15.82:0/1911324428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 0x7f2bec0cf390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0cf8d0 tx=0x7f2be002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2de754f4a590045 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:20.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.369+0000 7f2beb400640 1 -- 172.21.15.82:0/1911324428 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0031870 con 0x7f2bec15e0d0 2024-03-16T17:58:20.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.369+0000 7f2beb400640 1 -- 172.21.15.82:0/1911324428 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be00319d0 con 0x7f2bec15e0d0 2024-03-16T17:58:20.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.369+0000 7f2beb400640 1 -- 172.21.15.82:0/1911324428 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0031d60 con 0x7f2bec15e0d0 2024-03-16T17:58:20.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.369+0000 7f2bf3800640 1 -- 172.21.15.82:0/1911324428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 msgr2=0x7f2bec0cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:20.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.369+0000 7f2bf3800640 1 --2- 172.21.15.82:0/1911324428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 0x7f2bec0cf390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0cf8d0 tx=0x7f2be002b910 comp rx=0 tx=0).stop 2024-03-16T17:58:20.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.370+0000 7f2bf3800640 1 -- 172.21.15.82:0/1911324428 shutdown_connections 2024-03-16T17:58:20.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.370+0000 7f2bf3800640 1 --2- 172.21.15.82:0/1911324428 >> v2:172.21.15.90:3300/0 conn(0x7f2bec0d1c00 0x7f2bec0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.370+0000 7f2bf3800640 1 --2- 172.21.15.82:0/1911324428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 0x7f2bec0cf390 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.370+0000 7f2bf3800640 1 -- 172.21.15.82:0/1911324428 >> 172.21.15.82:0/1911324428 conn(0x7f2bec0d3dc0 msgr2=0x7f2bec0d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:20.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.370+0000 7f2bf3800640 1 -- 172.21.15.82:0/1911324428 shutdown_connections 2024-03-16T17:58:20.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.370+0000 7f2bf3800640 1 -- 172.21.15.82:0/1911324428 wait complete. 2024-03-16T17:58:20.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.372+0000 7f2bf3800640 1 Processor -- start 2024-03-16T17:58:20.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.372+0000 7f2bf3800640 1 -- start start 2024-03-16T17:58:20.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf3800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bec0d1c00 0x7f2bec0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf3800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2bec0d28c0 0x7f2bec1f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf3800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 0x7f2bec160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf3800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1f7e60 con 0x7f2bec15e0d0 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf3800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec15f370 con 0x7f2bec0d1c00 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf3800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec15e760 con 0x7f2bec0d28c0 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf1800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 0x7f2bec160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf1800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 0x7f2bec160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47162/0 (socket says 172.21.15.82:47162) 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf1800640 1 -- 172.21.15.82:0/1941535154 learned_addr learned my addr 172.21.15.82:0/1941535154 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf0e00640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bec0d1c00 0x7f2bec0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bebe00640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2bec0d28c0 0x7f2bec1f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf1800640 1 -- 172.21.15.82:0/1941535154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2bec0d28c0 msgr2=0x7f2bec1f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf1800640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2bec0d28c0 0x7f2bec1f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf1800640 1 -- 172.21.15.82:0/1941535154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bec0d1c00 msgr2=0x7f2bec0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:20.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf1800640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bec0d1c00 0x7f2bec0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bf1800640 1 -- 172.21.15.82:0/1941535154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be00094d0 con 0x7f2bec15e0d0 2024-03-16T17:58:20.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.373+0000 7f2bebe00640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2bec0d28c0 0x7f2bec1f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:20.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.374+0000 7f2bf1800640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 0x7f2bec160160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2be400c7b0 tx=0x7f2be400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:20.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.374+0000 7f2be8c00640 1 -- 172.21.15.82:0/1941535154 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be4004b60 con 0x7f2bec15e0d0 2024-03-16T17:58:20.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.374+0000 7f2be8c00640 1 -- 172.21.15.82:0/1941535154 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be4004cc0 con 0x7f2bec15e0d0 2024-03-16T17:58:20.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.374+0000 7f2bf3800640 1 -- 172.21.15.82:0/1941535154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec15ea40 con 0x7f2bec15e0d0 2024-03-16T17:58:20.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.374+0000 7f2be8c00640 1 -- 172.21.15.82:0/1941535154 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be400aa90 con 0x7f2bec15e0d0 2024-03-16T17:58:20.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.374+0000 7f2bf3800640 1 -- 172.21.15.82:0/1941535154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec15f000 con 0x7f2bec15e0d0 2024-03-16T17:58:20.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.376+0000 7f2bf3800640 1 -- 172.21.15.82:0/1941535154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec15f500 con 0x7f2bec15e0d0 2024-03-16T17:58:20.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.376+0000 7f2be8c00640 1 -- 172.21.15.82:0/1941535154 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f2be400abf0 con 0x7f2bec15e0d0 2024-03-16T17:58:20.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.377+0000 7f2be8c00640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2bcc06b3b0 0x7f2bcc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.377+0000 7f2be8c00640 1 -- 172.21.15.82:0/1941535154 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2be4089d80 con 0x7f2bec15e0d0 2024-03-16T17:58:20.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.378+0000 7f2bf0e00640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2bcc06b3b0 0x7f2bcc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.378+0000 7f2bf0e00640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2bcc06b3b0 0x7f2bcc06d870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2be0036000 tx=0x7f2be003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:20.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.382+0000 7f2be8c00640 1 -- 172.21.15.82:0/1941535154 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2be40549d0 con 0x7f2bec15e0d0 2024-03-16T17:58:20.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.581+0000 7f2bf3800640 1 -- 172.21.15.82:0/1941535154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bec068640 con 0x7f2bec15e0d0 2024-03-16T17:58:20.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.582+0000 7f2be8c00640 1 -- 172.21.15.82:0/1941535154 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2be4058090 con 0x7f2bec15e0d0 2024-03-16T17:58:20.586 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:58:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 -- 172.21.15.82:0/1941535154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2bcc06b3b0 msgr2=0x7f2bcc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2bcc06b3b0 0x7f2bcc06d870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2be0036000 tx=0x7f2be003c040 comp rx=0 tx=0).stop 2024-03-16T17:58:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 -- 172.21.15.82:0/1941535154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 msgr2=0x7f2bec160160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 0x7f2bec160160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2be400c7b0 tx=0x7f2be400cc70 comp rx=0 tx=0).stop 2024-03-16T17:58:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 -- 172.21.15.82:0/1941535154 shutdown_connections 2024-03-16T17:58:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2bcc06b3b0 0x7f2bcc06d870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec15e0d0 0x7f2bec160160 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2bec0d28c0 0x7f2bec1f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 --2- 172.21.15.82:0/1941535154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2bec0d1c00 0x7f2bec0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 -- 172.21.15.82:0/1941535154 >> 172.21.15.82:0/1941535154 conn(0x7f2bec0d3dc0 msgr2=0x7f2bec0b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:20.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 -- 172.21.15.82:0/1941535154 shutdown_connections 2024-03-16T17:58:20.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.585+0000 7f2bf3800640 1 -- 172.21.15.82:0/1941535154 wait complete. 2024-03-16T17:58:20.608 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:58:20.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.821+0000 7f0413e00640 1 Processor -- start 2024-03-16T17:58:20.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.821+0000 7f0413e00640 1 -- start start 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.822+0000 7f0413e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f04140d0ba0 0x7f04140d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.822+0000 7f0413e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f04140b28d0 0x7f04140b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.822+0000 7f0413e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04140d1790 con 0x7f041415dd40 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.823+0000 7f0413e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140d1a70 con 0x7f04140d0ba0 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.823+0000 7f0413e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04140cf410 con 0x7f04140b28d0 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.823+0000 7f0412000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f04140d0ba0 0x7f04140d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.823+0000 7f0412000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f04140d0ba0 0x7f04140d46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47170/0 (socket says 172.21.15.82:47170) 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.823+0000 7f0412000640 1 -- 172.21.15.82:0/503608137 learned_addr learned my addr 172.21.15.82:0/503608137 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.823+0000 7f0413400640 1 --2- 172.21.15.82:0/503608137 >> v2:172.21.15.90:3300/0 conn(0x7f04140b28d0 0x7f04140b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.823+0000 7f0412000640 1 -- 172.21.15.82:0/503608137 >> v2:172.21.15.90:3300/0 conn(0x7f04140b28d0 msgr2=0x7f04140b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.823+0000 7f0412000640 1 --2- 172.21.15.82:0/503608137 >> v2:172.21.15.90:3300/0 conn(0x7f04140b28d0 0x7f04140b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.823+0000 7f0412000640 1 -- 172.21.15.82:0/503608137 >> v1:172.21.15.90:6789/0 conn(0x7f041415dd40 legacy=0x7f04140d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:20.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.823+0000 7f0412000640 1 -- 172.21.15.82:0/503608137 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140cf6b0 con 0x7f04140d0ba0 2024-03-16T17:58:20.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.825+0000 7f0412000640 1 --2- 172.21.15.82:0/503608137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04140d0ba0 0x7f04140d46e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f04140ce1a0 tx=0x7f040002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f759c73ef0eca511 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:20.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.825+0000 7f0411600640 1 -- 172.21.15.82:0/503608137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04000317e0 con 0x7f04140d0ba0 2024-03-16T17:58:20.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.825+0000 7f0411600640 1 -- 172.21.15.82:0/503608137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0400031940 con 0x7f04140d0ba0 2024-03-16T17:58:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.825+0000 7f0411600640 1 -- 172.21.15.82:0/503608137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0400031cd0 con 0x7f04140d0ba0 2024-03-16T17:58:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.825+0000 7f0413e00640 1 -- 172.21.15.82:0/503608137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04140d0ba0 msgr2=0x7f04140d46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:20.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.825+0000 7f0413e00640 1 --2- 172.21.15.82:0/503608137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04140d0ba0 0x7f04140d46e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f04140ce1a0 tx=0x7f040002b880 comp rx=0 tx=0).stop 2024-03-16T17:58:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.826+0000 7f0413e00640 1 -- 172.21.15.82:0/503608137 shutdown_connections 2024-03-16T17:58:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.826+0000 7f0413e00640 1 --2- 172.21.15.82:0/503608137 >> v2:172.21.15.90:3300/0 conn(0x7f04140b28d0 0x7f04140b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.826+0000 7f0413e00640 1 --2- 172.21.15.82:0/503608137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04140d0ba0 0x7f04140d46e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.826+0000 7f0413e00640 1 -- 172.21.15.82:0/503608137 >> 172.21.15.82:0/503608137 conn(0x7f04140d3bc0 msgr2=0x7f04140d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.826+0000 7f0413e00640 1 -- 172.21.15.82:0/503608137 shutdown_connections 2024-03-16T17:58:20.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.826+0000 7f0413e00640 1 -- 172.21.15.82:0/503608137 wait complete. 2024-03-16T17:58:20.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.827+0000 7f0413e00640 1 Processor -- start 2024-03-16T17:58:20.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.827+0000 7f0413e00640 1 -- start start 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.828+0000 7f0413e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f04140b28d0 0x7f0414155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.828+0000 7f0413e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04140d0ba0 0x7f041415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.828+0000 7f0413e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f041415dd40 0x7f0414159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.828+0000 7f0413e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04140cd960 con 0x7f041415dd40 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.828+0000 7f0413e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041415e430 con 0x7f04140d0ba0 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.828+0000 7f0413e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04140b73f0 con 0x7f04140b28d0 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0413400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f041415dd40 0x7f0414159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0413400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f041415dd40 0x7f0414159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47174/0 (socket says 172.21.15.82:47174) 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0413400640 1 -- 172.21.15.82:0/1866233573 learned_addr learned my addr 172.21.15.82:0/1866233573 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0412000640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04140d0ba0 0x7f041415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0412a00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f04140b28d0 0x7f0414155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0413400640 1 -- 172.21.15.82:0/1866233573 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f04140b28d0 msgr2=0x7f0414155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0413400640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f04140b28d0 0x7f0414155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0413400640 1 -- 172.21.15.82:0/1866233573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04140d0ba0 msgr2=0x7f041415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0413400640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04140d0ba0 0x7f041415ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0413400640 1 -- 172.21.15.82:0/1866233573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04000094d0 con 0x7f041415dd40 2024-03-16T17:58:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0412a00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f04140b28d0 0x7f0414155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:20.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.829+0000 7f0413400640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f041415dd40 0x7f0414159460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f040c007e40 tx=0x7f040c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:20.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.830+0000 7f040aa00640 1 -- 172.21.15.82:0/1866233573 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c014620 con 0x7f041415dd40 2024-03-16T17:58:20.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.830+0000 7f040aa00640 1 -- 172.21.15.82:0/1866233573 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c00ee90 con 0x7f041415dd40 2024-03-16T17:58:20.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.830+0000 7f0413e00640 1 -- 172.21.15.82:0/1866233573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041415e2a0 con 0x7f041415dd40 2024-03-16T17:58:20.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.830+0000 7f040aa00640 1 -- 172.21.15.82:0/1866233573 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c0149e0 con 0x7f041415dd40 2024-03-16T17:58:20.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.830+0000 7f0413e00640 1 -- 172.21.15.82:0/1866233573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041415e680 con 0x7f041415dd40 2024-03-16T17:58:20.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.831+0000 7f0413e00640 1 -- 172.21.15.82:0/1866233573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041404f250 con 0x7f041415dd40 2024-03-16T17:58:20.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.832+0000 7f040aa00640 1 -- 172.21.15.82:0/1866233573 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f040c014b40 con 0x7f041415dd40 2024-03-16T17:58:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.833+0000 7f040aa00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f03f406b3b0 0x7f03f406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.833+0000 7f040aa00640 1 -- 172.21.15.82:0/1866233573 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f040c08a630 con 0x7f041415dd40 2024-03-16T17:58:20.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.833+0000 7f0412a00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f03f406b3b0 0x7f03f406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:20.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.834+0000 7f0412a00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f03f406b3b0 0x7f03f406d870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f04140b8300 tx=0x7f0404008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:20.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:20.837+0000 7f040aa00640 1 -- 172.21.15.82:0/1866233573 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f040c0551f0 con 0x7f041415dd40 2024-03-16T17:58:21.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.033+0000 7f0413e00640 1 -- 172.21.15.82:0/1866233573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f04140d1ee0 con 0x7f041415dd40 2024-03-16T17:58:21.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.034+0000 7f040aa00640 1 -- 172.21.15.82:0/1866233573 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f04140d1ee0 con 0x7f041415dd40 2024-03-16T17:58:21.038 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:58:21.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 -- 172.21.15.82:0/1866233573 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f03f406b3b0 msgr2=0x7f03f406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f03f406b3b0 0x7f03f406d870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f04140b8300 tx=0x7f0404008040 comp rx=0 tx=0).stop 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 -- 172.21.15.82:0/1866233573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f041415dd40 msgr2=0x7f0414159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f041415dd40 0x7f0414159460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f040c007e40 tx=0x7f040c00e4b0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 -- 172.21.15.82:0/1866233573 shutdown_connections 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f03f406b3b0 0x7f03f406d870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f041415dd40 0x7f0414159460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04140d0ba0 0x7f041415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 --2- 172.21.15.82:0/1866233573 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f04140b28d0 0x7f0414155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 -- 172.21.15.82:0/1866233573 >> 172.21.15.82:0/1866233573 conn(0x7f04140d3bc0 msgr2=0x7f041406b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 -- 172.21.15.82:0/1866233573 shutdown_connections 2024-03-16T17:58:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.037+0000 7f0413e00640 1 -- 172.21.15.82:0/1866233573 wait complete. 2024-03-16T17:58:21.059 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-03-16T17:58:21.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.268+0000 7f81a6a00640 1 Processor -- start 2024-03-16T17:58:21.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.268+0000 7f81a6a00640 1 -- start start 2024-03-16T17:58:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f81a6a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f81a01454d0 0x7f81a00b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f81a6a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81a00cdf80 0x7f81a00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f81a6a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f81a00cfe70 0x7f81a015f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f81a6a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a00d0320 con 0x7f81a00cfe70 2024-03-16T17:58:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f81a6a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a00b4b90 con 0x7f81a01454d0 2024-03-16T17:58:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f81a6a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00cf500 con 0x7f81a00cdf80 2024-03-16T17:58:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f81a4a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f81a00cfe70 0x7f81a015f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f81a4a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f81a00cfe70 0x7f81a015f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47190/0 (socket says 172.21.15.82:47190) 2024-03-16T17:58:21.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f81a4a00640 1 -- 172.21.15.82:0/608812375 learned_addr learned my addr 172.21.15.82:0/608812375 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:21.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f819fe00640 1 --2- 172.21.15.82:0/608812375 >> v2:172.21.15.90:3300/0 conn(0x7f81a01454d0 0x7f81a00b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:21.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.270+0000 7f819f400640 1 --2- 172.21.15.82:0/608812375 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81a00cdf80 0x7f81a00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:21.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.271+0000 7f81a4a00640 1 -- 172.21.15.82:0/608812375 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81a00cdf80 msgr2=0x7f81a00cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:21.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.271+0000 7f81a4a00640 1 --2- 172.21.15.82:0/608812375 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81a00cdf80 0x7f81a00cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.271+0000 7f81a4a00640 1 -- 172.21.15.82:0/608812375 >> v2:172.21.15.90:3300/0 conn(0x7f81a01454d0 msgr2=0x7f81a00b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:21.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.271+0000 7f81a4a00640 1 --2- 172.21.15.82:0/608812375 >> v2:172.21.15.90:3300/0 conn(0x7f81a01454d0 0x7f81a00b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.271+0000 7f81a4a00640 1 -- 172.21.15.82:0/608812375 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a01458b0 con 0x7f81a00cfe70 2024-03-16T17:58:21.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.272+0000 7f81a4a00640 1 --2- 172.21.15.82:0/608812375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cfe70 0x7f81a015f8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f81a00b2800 tx=0x7f819802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4324f6bf53292f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.272+0000 7f819ea00640 1 -- 172.21.15.82:0/608812375 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819802ca70 con 0x7f81a00cfe70 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.272+0000 7f819ea00640 1 -- 172.21.15.82:0/608812375 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819802cbd0 con 0x7f81a00cfe70 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.272+0000 7f819ea00640 1 -- 172.21.15.82:0/608812375 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81980343f0 con 0x7f81a00cfe70 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.273+0000 7f81a6a00640 1 -- 172.21.15.82:0/608812375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cfe70 msgr2=0x7f81a015f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.273+0000 7f81a6a00640 1 --2- 172.21.15.82:0/608812375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cfe70 0x7f81a015f8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f81a00b2800 tx=0x7f819802bed0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.273+0000 7f81a6a00640 1 -- 172.21.15.82:0/608812375 shutdown_connections 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.273+0000 7f81a6a00640 1 --2- 172.21.15.82:0/608812375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cfe70 0x7f81a015f8b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.273+0000 7f81a6a00640 1 --2- 172.21.15.82:0/608812375 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81a00cdf80 0x7f81a00cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.273+0000 7f81a6a00640 1 --2- 172.21.15.82:0/608812375 >> v2:172.21.15.90:3300/0 conn(0x7f81a01454d0 0x7f81a00b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.273+0000 7f81a6a00640 1 -- 172.21.15.82:0/608812375 >> 172.21.15.82:0/608812375 conn(0x7f81a006ae70 msgr2=0x7f81a006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.273+0000 7f81a6a00640 1 -- 172.21.15.82:0/608812375 shutdown_connections 2024-03-16T17:58:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.273+0000 7f81a6a00640 1 -- 172.21.15.82:0/608812375 wait complete. 2024-03-16T17:58:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.275+0000 7f81a6a00640 1 Processor -- start 2024-03-16T17:58:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.275+0000 7f81a6a00640 1 -- start start 2024-03-16T17:58:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.276+0000 7f81a6a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cdf80 0x7f81a014d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.276+0000 7f81a6a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a00cfe70 0x7f81a013ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.276+0000 7f81a6a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81a01454d0 0x7f81a013cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.276+0000 7f81a6a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a013edc0 con 0x7f81a00cdf80 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.276+0000 7f81a6a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a014d660 con 0x7f81a00cfe70 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.276+0000 7f81a6a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0140570 con 0x7f81a01454d0 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f819fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cdf80 0x7f81a014d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f819fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cdf80 0x7f81a014d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47206/0 (socket says 172.21.15.82:47206) 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f819fe00640 1 -- 172.21.15.82:0/2078228372 learned_addr learned my addr 172.21.15.82:0/2078228372 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f819f400640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a00cfe70 0x7f81a013ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f81a4a00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81a01454d0 0x7f81a013cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f819fe00640 1 -- 172.21.15.82:0/2078228372 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81a01454d0 msgr2=0x7f81a013cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f819fe00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81a01454d0 0x7f81a013cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f819fe00640 1 -- 172.21.15.82:0/2078228372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a00cfe70 msgr2=0x7f81a013ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f819fe00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a00cfe70 0x7f81a013ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f819fe00640 1 -- 172.21.15.82:0/2078228372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198009b30 con 0x7f81a00cdf80 2024-03-16T17:58:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.277+0000 7f819f400640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a00cfe70 0x7f81a013ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.278+0000 7f819fe00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cdf80 0x7f81a014d120 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f819000d800 tx=0x7f819000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.278+0000 7f8197e00640 1 -- 172.21.15.82:0/2078228372 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819000b6c0 con 0x7f81a00cdf80 2024-03-16T17:58:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.278+0000 7f8197e00640 1 -- 172.21.15.82:0/2078228372 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819000b820 con 0x7f81a00cdf80 2024-03-16T17:58:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.278+0000 7f8197e00640 1 -- 172.21.15.82:0/2078228372 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819000bbb0 con 0x7f81a00cdf80 2024-03-16T17:58:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.278+0000 7f81a6a00640 1 -- 172.21.15.82:0/2078228372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a0140850 con 0x7f81a00cdf80 2024-03-16T17:58:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.278+0000 7f81a6a00640 1 -- 172.21.15.82:0/2078228372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a0140e10 con 0x7f81a00cdf80 2024-03-16T17:58:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.279+0000 7f81a6a00640 1 -- 172.21.15.82:0/2078228372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a0067f30 con 0x7f81a00cdf80 2024-03-16T17:58:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.280+0000 7f8197e00640 1 -- 172.21.15.82:0/2078228372 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f819000bd10 con 0x7f81a00cdf80 2024-03-16T17:58:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.281+0000 7f8197e00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f818006b3b0 0x7f818006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.281+0000 7f8197e00640 1 -- 172.21.15.82:0/2078228372 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f819008a210 con 0x7f81a00cdf80 2024-03-16T17:58:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.281+0000 7f819f400640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f818006b3b0 0x7f818006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.282+0000 7f819f400640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f818006b3b0 0x7f818006d870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f81a013c1c0 tx=0x7f818c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:21.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.286+0000 7f8197e00640 1 -- 172.21.15.82:0/2078228372 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8190054dd0 con 0x7f81a00cdf80 2024-03-16T17:58:21.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:21.556+0000 7f81a6a00640 1 -- 172.21.15.82:0/2078228372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) v1 -- 0x7f81a00b4640 con 0x7f81a00cdf80 2024-03-16T17:58:22.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.188+0000 7f8197e00640 1 -- 172.21.15.82:0/2078228372 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v56) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8190058490 con 0x7f81a00cdf80 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 -- 172.21.15.82:0/2078228372 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f818006b3b0 msgr2=0x7f818006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f818006b3b0 0x7f818006d870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f81a013c1c0 tx=0x7f818c008040 comp rx=0 tx=0).stop 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 -- 172.21.15.82:0/2078228372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cdf80 msgr2=0x7f81a014d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cdf80 0x7f81a014d120 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f819000d800 tx=0x7f819000dcc0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 -- 172.21.15.82:0/2078228372 shutdown_connections 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f818006b3b0 0x7f818006d870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81a01454d0 0x7f81a013cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a00cfe70 0x7f81a013ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 --2- 172.21.15.82:0/2078228372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a00cdf80 0x7f81a014d120 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 -- 172.21.15.82:0/2078228372 >> 172.21.15.82:0/2078228372 conn(0x7f81a006ae70 msgr2=0x7f81a0160020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.191+0000 7f81a6a00640 1 -- 172.21.15.82:0/2078228372 shutdown_connections 2024-03-16T17:58:22.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.192+0000 7f81a6a00640 1 -- 172.21.15.82:0/2078228372 wait complete. 2024-03-16T17:58:22.214 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-16T17:58:22.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.424+0000 7fe4f7800640 1 Processor -- start 2024-03-16T17:58:22.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.424+0000 7fe4f7800640 1 -- start start 2024-03-16T17:58:22.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.425+0000 7fe4f7800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f0144330 0x7fe4f0070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:22.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f7800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe4f00cc380 0x7fe4f00cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:22.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f7800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4f00ce8f0 con 0x7fe4f00714f0 2024-03-16T17:58:22.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f7800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4f00ccd50 con 0x7fe4f00cc380 2024-03-16T17:58:22.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f7800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f015f2d0 con 0x7fe4f0144330 2024-03-16T17:58:22.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f5800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe4f00cc380 0x7fe4f00cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:22.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f5800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe4f00cc380 0x7fe4f00cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47214/0 (socket says 172.21.15.82:47214) 2024-03-16T17:58:22.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f5800640 1 -- 172.21.15.82:0/2569786861 learned_addr learned my addr 172.21.15.82:0/2569786861 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:22.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f4e00640 1 --2- 172.21.15.82:0/2569786861 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f0144330 0x7fe4f0070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:22.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f5800640 1 -- 172.21.15.82:0/2569786861 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f0144330 msgr2=0x7fe4f0070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:22.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f5800640 1 --2- 172.21.15.82:0/2569786861 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f0144330 0x7fe4f0070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f5800640 1 -- 172.21.15.82:0/2569786861 >> v1:172.21.15.90:6789/0 conn(0x7fe4f00714f0 legacy=0x7fe4f015e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:22.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.426+0000 7fe4f5800640 1 -- 172.21.15.82:0/2569786861 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f015f5b0 con 0x7fe4f00cc380 2024-03-16T17:58:22.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.427+0000 7fe4f5800640 1 --2- 172.21.15.82:0/2569786861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f00cc380 0x7fe4f00cc810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009e50 tx=0x7fe4e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da65000c503bd99f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:22.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.428+0000 7fe4ef400640 1 -- 172.21.15.82:0/2569786861 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e802ca70 con 0x7fe4f00cc380 2024-03-16T17:58:22.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.428+0000 7fe4ef400640 1 -- 172.21.15.82:0/2569786861 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e802cbd0 con 0x7fe4f00cc380 2024-03-16T17:58:22.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.428+0000 7fe4ef400640 1 -- 172.21.15.82:0/2569786861 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e80343f0 con 0x7fe4f00cc380 2024-03-16T17:58:22.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.428+0000 7fe4f7800640 1 -- 172.21.15.82:0/2569786861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f00cc380 msgr2=0x7fe4f00cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:22.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.428+0000 7fe4f7800640 1 --2- 172.21.15.82:0/2569786861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f00cc380 0x7fe4f00cc810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009e50 tx=0x7fe4e802bed0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.429+0000 7fe4f7800640 1 -- 172.21.15.82:0/2569786861 shutdown_connections 2024-03-16T17:58:22.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.429+0000 7fe4f7800640 1 --2- 172.21.15.82:0/2569786861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f00cc380 0x7fe4f00cc810 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.429+0000 7fe4f7800640 1 --2- 172.21.15.82:0/2569786861 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f0144330 0x7fe4f0070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.429+0000 7fe4f7800640 1 -- 172.21.15.82:0/2569786861 >> 172.21.15.82:0/2569786861 conn(0x7fe4f00b1640 msgr2=0x7fe4f00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:22.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.429+0000 7fe4f7800640 1 -- 172.21.15.82:0/2569786861 shutdown_connections 2024-03-16T17:58:22.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.429+0000 7fe4f7800640 1 -- 172.21.15.82:0/2569786861 wait complete. 2024-03-16T17:58:22.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.431+0000 7fe4f7800640 1 Processor -- start 2024-03-16T17:58:22.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.431+0000 7fe4f7800640 1 -- start start 2024-03-16T17:58:22.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.432+0000 7fe4f7800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f00714f0 0x7fe4f013f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:22.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.432+0000 7fe4f7800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f00cc380 0x7fe4f01efe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:22.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.432+0000 7fe4f7800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0144330 0x7fe4f01f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:22.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.432+0000 7fe4f7800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f0140860 con 0x7fe4f0144330 2024-03-16T17:58:22.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.432+0000 7fe4f7800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f01f03a0 con 0x7fe4f00cc380 2024-03-16T17:58:22.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.432+0000 7fe4f7800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4f013fc50 con 0x7fe4f00714f0 2024-03-16T17:58:22.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.432+0000 7fe4f5800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0144330 0x7fe4f01f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:22.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4f5800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0144330 0x7fe4f01f29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47222/0 (socket says 172.21.15.82:47222) 2024-03-16T17:58:22.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4f5800640 1 -- 172.21.15.82:0/3842343915 learned_addr learned my addr 172.21.15.82:0/3842343915 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:22.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4f4e00640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f00714f0 0x7fe4f013f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4efe00640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f00cc380 0x7fe4f01efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4f5800640 1 -- 172.21.15.82:0/3842343915 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f00714f0 msgr2=0x7fe4f013f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4f5800640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f00714f0 0x7fe4f013f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4f5800640 1 -- 172.21.15.82:0/3842343915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f00cc380 msgr2=0x7fe4f01efe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4f5800640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f00cc380 0x7fe4f01efe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4f5800640 1 -- 172.21.15.82:0/3842343915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e8009b30 con 0x7fe4f0144330 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4f4e00640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f00714f0 0x7fe4f013f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.433+0000 7fe4f5800640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0144330 0x7fe4f01f29d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8036040 tx=0x7fe4e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.434+0000 7fe4ecc00640 1 -- 172.21.15.82:0/3842343915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8009510 con 0x7fe4f0144330 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.434+0000 7fe4ecc00640 1 -- 172.21.15.82:0/3842343915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e803e450 con 0x7fe4f0144330 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.434+0000 7fe4f7800640 1 -- 172.21.15.82:0/3842343915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4f013ff30 con 0x7fe4f0144330 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.434+0000 7fe4ecc00640 1 -- 172.21.15.82:0/3842343915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e803e7e0 con 0x7fe4f0144330 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.434+0000 7fe4f7800640 1 -- 172.21.15.82:0/3842343915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4f01404f0 con 0x7fe4f0144330 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.435+0000 7fe4f7800640 1 -- 172.21.15.82:0/3842343915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4f0067f30 con 0x7fe4f0144330 2024-03-16T17:58:22.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.436+0000 7fe4ecc00640 1 -- 172.21.15.82:0/3842343915 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe4e803e940 con 0x7fe4f0144330 2024-03-16T17:58:22.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.437+0000 7fe4ecc00640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe4d006b3e0 0x7fe4d006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:22.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.437+0000 7fe4ecc00640 1 -- 172.21.15.82:0/3842343915 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4e80ac6a0 con 0x7fe4f0144330 2024-03-16T17:58:22.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.437+0000 7fe4f4e00640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe4d006b3e0 0x7fe4d006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:22.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.438+0000 7fe4f4e00640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe4d006b3e0 0x7fe4d006d8a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4006fd0 tx=0x7fe4e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:22.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.441+0000 7fe4ecc00640 1 -- 172.21.15.82:0/3842343915 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe4e8077300 con 0x7fe4f0144330 2024-03-16T17:58:22.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.639+0000 7fe4f7800640 1 -- 172.21.15.82:0/3842343915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe4f00ca540 con 0x7fe4f0144330 2024-03-16T17:58:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.644+0000 7fe4ecc00640 1 -- 172.21.15.82:0/3842343915 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe4e807a9c0 con 0x7fe4f0144330 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 -- 172.21.15.82:0/3842343915 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe4d006b3e0 msgr2=0x7fe4d006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe4d006b3e0 0x7fe4d006d8a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4006fd0 tx=0x7fe4e4008040 comp rx=0 tx=0).stop 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 -- 172.21.15.82:0/3842343915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0144330 msgr2=0x7fe4f01f29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0144330 0x7fe4f01f29d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8036040 tx=0x7fe4e8004a40 comp rx=0 tx=0).stop 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 -- 172.21.15.82:0/3842343915 shutdown_connections 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe4d006b3e0 0x7fe4d006d8a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4f0144330 0x7fe4f01f29d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4f00cc380 0x7fe4f01efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 --2- 172.21.15.82:0/3842343915 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4f00714f0 0x7fe4f013f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 -- 172.21.15.82:0/3842343915 >> 172.21.15.82:0/3842343915 conn(0x7fe4f00b1640 msgr2=0x7fe4f015e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 -- 172.21.15.82:0/3842343915 shutdown_connections 2024-03-16T17:58:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:22.647+0000 7fe4f7800640 1 -- 172.21.15.82:0/3842343915 wait complete. 2024-03-16T17:58:22.671 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-16T17:58:22.671 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T17:58:22.781 DEBUG:teuthology.orchestra.run.smithi090:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T17:58:22.900 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-16T17:58:22.901 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:58:22.901 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:22.959 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:58:22.959 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:23.009 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:58:23.009 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:23.059 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:23.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.317+0000 7f1b9c400640 1 Processor -- start 2024-03-16T17:58:23.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.317+0000 7f1b9c400640 1 -- start start 2024-03-16T17:58:23.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b9c400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1b941454d0 0x7f1b940b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b9c400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1b940cdf80 0x7f1b940cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b9c400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940cfe70 0x7f1b9415f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b9c400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b940d0300 con 0x7f1b940cdf80 2024-03-16T17:58:23.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b9c400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b940b4b90 con 0x7f1b941454d0 2024-03-16T17:58:23.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b9c400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940cf500 con 0x7f1b940cfe70 2024-03-16T17:58:23.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b99000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1b940cdf80 0x7f1b940cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b99000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1b940cdf80 0x7f1b940cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47232/0 (socket says 172.21.15.82:47232) 2024-03-16T17:58:23.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b99000640 1 -- 172.21.15.82:0/2776943285 learned_addr learned my addr 172.21.15.82:0/2776943285 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:23.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b9a400640 1 --2- 172.21.15.82:0/2776943285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940cfe70 0x7f1b9415f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.319+0000 7f1b99a00640 1 --2- 172.21.15.82:0/2776943285 >> v2:172.21.15.90:3300/0 conn(0x7f1b941454d0 0x7f1b940b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.320+0000 7f1b99000640 1 -- 172.21.15.82:0/2776943285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940cfe70 msgr2=0x7f1b9415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:23.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.320+0000 7f1b99000640 1 --2- 172.21.15.82:0/2776943285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940cfe70 0x7f1b9415f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.320+0000 7f1b99000640 1 -- 172.21.15.82:0/2776943285 >> v2:172.21.15.90:3300/0 conn(0x7f1b941454d0 msgr2=0x7f1b940b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:23.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.320+0000 7f1b99000640 1 --2- 172.21.15.82:0/2776943285 >> v2:172.21.15.90:3300/0 conn(0x7f1b941454d0 0x7f1b940b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.320+0000 7f1b99000640 1 -- 172.21.15.82:0/2776943285 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b941458b0 con 0x7f1b940cdf80 2024-03-16T17:58:23.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.321+0000 7f1b99000640 1 --2- 172.21.15.82:0/2776943285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cdf80 0x7f1b940cf930 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b940b2800 tx=0x7f1b8002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9436fb32fa721951 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:23.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.321+0000 7f1b93e00640 1 -- 172.21.15.82:0/2776943285 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80031860 con 0x7f1b940cdf80 2024-03-16T17:58:23.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.321+0000 7f1b93e00640 1 -- 172.21.15.82:0/2776943285 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b800319c0 con 0x7f1b940cdf80 2024-03-16T17:58:23.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.321+0000 7f1b93e00640 1 -- 172.21.15.82:0/2776943285 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80031d50 con 0x7f1b940cdf80 2024-03-16T17:58:23.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.322+0000 7f1b9c400640 1 -- 172.21.15.82:0/2776943285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cdf80 msgr2=0x7f1b940cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.322+0000 7f1b9c400640 1 --2- 172.21.15.82:0/2776943285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cdf80 0x7f1b940cf930 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1b940b2800 tx=0x7f1b8002b900 comp rx=0 tx=0).stop 2024-03-16T17:58:23.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.322+0000 7f1b9c400640 1 -- 172.21.15.82:0/2776943285 shutdown_connections 2024-03-16T17:58:23.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.322+0000 7f1b9c400640 1 --2- 172.21.15.82:0/2776943285 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b940cfe70 0x7f1b9415f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.322+0000 7f1b9c400640 1 --2- 172.21.15.82:0/2776943285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cdf80 0x7f1b940cf930 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.322+0000 7f1b9c400640 1 --2- 172.21.15.82:0/2776943285 >> v2:172.21.15.90:3300/0 conn(0x7f1b941454d0 0x7f1b940b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.322+0000 7f1b9c400640 1 -- 172.21.15.82:0/2776943285 >> 172.21.15.82:0/2776943285 conn(0x7f1b9406ae70 msgr2=0x7f1b9406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:23.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.322+0000 7f1b9c400640 1 -- 172.21.15.82:0/2776943285 shutdown_connections 2024-03-16T17:58:23.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.322+0000 7f1b9c400640 1 -- 172.21.15.82:0/2776943285 wait complete. 2024-03-16T17:58:23.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.324+0000 7f1b9c400640 1 Processor -- start 2024-03-16T17:58:23.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.324+0000 7f1b9c400640 1 -- start start 2024-03-16T17:58:23.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b9c400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b940cdf80 0x7f1b940a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b9c400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cfe70 0x7f1b941f46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b9c400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b941454d0 0x7f1b9415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b9c400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940aaba0 con 0x7f1b940cfe70 2024-03-16T17:58:23.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b9c400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b941f4c20 con 0x7f1b940cdf80 2024-03-16T17:58:23.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b9c400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b940a9b00 con 0x7f1b941454d0 2024-03-16T17:58:23.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b99000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cfe70 0x7f1b941f46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b99000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cfe70 0x7f1b941f46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47234/0 (socket says 172.21.15.82:47234) 2024-03-16T17:58:23.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b99000640 1 -- 172.21.15.82:0/2583681974 learned_addr learned my addr 172.21.15.82:0/2583681974 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:23.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b99a00640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b940cdf80 0x7f1b940a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.325+0000 7f1b9a400640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b941454d0 0x7f1b9415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.326+0000 7f1b99000640 1 -- 172.21.15.82:0/2583681974 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b941454d0 msgr2=0x7f1b9415e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.326+0000 7f1b99000640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b941454d0 0x7f1b9415e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.326+0000 7f1b99000640 1 -- 172.21.15.82:0/2583681974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b940cdf80 msgr2=0x7f1b940a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.326+0000 7f1b99000640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b940cdf80 0x7f1b940a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.326+0000 7f1b99000640 1 -- 172.21.15.82:0/2583681974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800094d0 con 0x7f1b940cfe70 2024-03-16T17:58:23.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.326+0000 7f1b99000640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cfe70 0x7f1b941f46e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1b8002b940 tx=0x7f1b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:23.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.326+0000 7f1b91600640 1 -- 172.21.15.82:0/2583681974 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8003ad30 con 0x7f1b940cfe70 2024-03-16T17:58:23.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.326+0000 7f1b91600640 1 -- 172.21.15.82:0/2583681974 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b800443f0 con 0x7f1b940cfe70 2024-03-16T17:58:23.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.326+0000 7f1b9c400640 1 -- 172.21.15.82:0/2583681974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b940a9de0 con 0x7f1b940cfe70 2024-03-16T17:58:23.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.326+0000 7f1b91600640 1 -- 172.21.15.82:0/2583681974 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b800447a0 con 0x7f1b940cfe70 2024-03-16T17:58:23.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.327+0000 7f1b9c400640 1 -- 172.21.15.82:0/2583681974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b940aa3a0 con 0x7f1b940cfe70 2024-03-16T17:58:23.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.328+0000 7f1b9c400640 1 -- 172.21.15.82:0/2583681974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b94067f30 con 0x7f1b940cfe70 2024-03-16T17:58:23.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.328+0000 7f1b91600640 1 -- 172.21.15.82:0/2583681974 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f1b8004d450 con 0x7f1b940cfe70 2024-03-16T17:58:23.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.329+0000 7f1b91600640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1b7406b3b0 0x7f1b7406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.330+0000 7f1b91600640 1 -- 172.21.15.82:0/2583681974 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1b800ac1b0 con 0x7f1b940cfe70 2024-03-16T17:58:23.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.330+0000 7f1b99a00640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1b7406b3b0 0x7f1b7406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.330+0000 7f1b99a00640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1b7406b3b0 0x7f1b7406d870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b88006fd0 tx=0x7f1b88008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:23.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.334+0000 7f1b91600640 1 -- 172.21.15.82:0/2583681974 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1b80076eb0 con 0x7f1b940cfe70 2024-03-16T17:58:23.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.528+0000 7f1b9c400640 1 -- 172.21.15.82:0/2583681974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b940cef40 con 0x7f1b940cfe70 2024-03-16T17:58:23.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.531+0000 7f1b91600640 1 -- 172.21.15.82:0/2583681974 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f1b8007a570 con 0x7f1b940cfe70 2024-03-16T17:58:23.535 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:23.535 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:23.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.533+0000 7f1b9c400640 1 -- 172.21.15.82:0/2583681974 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1b7406b3b0 msgr2=0x7f1b7406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.533+0000 7f1b9c400640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1b7406b3b0 0x7f1b7406d870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b88006fd0 tx=0x7f1b88008040 comp rx=0 tx=0).stop 2024-03-16T17:58:23.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.533+0000 7f1b9c400640 1 -- 172.21.15.82:0/2583681974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cfe70 msgr2=0x7f1b941f46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.533+0000 7f1b9c400640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cfe70 0x7f1b941f46e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1b8002b940 tx=0x7f1b800046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.534+0000 7f1b9c400640 1 -- 172.21.15.82:0/2583681974 shutdown_connections 2024-03-16T17:58:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.534+0000 7f1b9c400640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1b7406b3b0 0x7f1b7406d870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.534+0000 7f1b9c400640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1b941454d0 0x7f1b9415e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.534+0000 7f1b9c400640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b940cfe70 0x7f1b941f46e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.534+0000 7f1b9c400640 1 --2- 172.21.15.82:0/2583681974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b940cdf80 0x7f1b940a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.534+0000 7f1b9c400640 1 -- 172.21.15.82:0/2583681974 >> 172.21.15.82:0/2583681974 conn(0x7f1b9406ae70 msgr2=0x7f1b9413ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.534+0000 7f1b9c400640 1 -- 172.21.15.82:0/2583681974 shutdown_connections 2024-03-16T17:58:23.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.534+0000 7f1b9c400640 1 -- 172.21.15.82:0/2583681974 wait complete. 2024-03-16T17:58:23.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:23.557 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:23.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.766+0000 7f7b41200640 1 Processor -- start 2024-03-16T17:58:23.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.767+0000 7f7b41200640 1 -- start start 2024-03-16T17:58:23.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.768+0000 7f7b41200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7b3c159960 0x7f7b3c06e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.768+0000 7f7b41200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3c15c370 con 0x7f7b3c0cca20 2024-03-16T17:58:23.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.768+0000 7f7b41200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3c15c650 con 0x7f7b3c06e770 2024-03-16T17:58:23.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.768+0000 7f7b41200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3c159d40 con 0x7f7b3c159960 2024-03-16T17:58:23.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.768+0000 7f7b39e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7b3c159960 0x7f7b3c06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.768+0000 7f7b3a800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7b3c0cca20 0x7f7b3c0cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47648/0 (socket says 172.21.15.82:47648) 2024-03-16T17:58:23.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.768+0000 7f7b3a800640 1 -- 172.21.15.82:0/2113180774 learned_addr learned my addr 172.21.15.82:0/2113180774 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:23.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.769+0000 7f7b39e00640 1 -- 172.21.15.82:0/2113180774 >> v1:172.21.15.90:6789/0 conn(0x7f7b3c06e770 legacy=0x7f7b3c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:23.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.769+0000 7f7b39e00640 1 -- 172.21.15.82:0/2113180774 >> v1:172.21.15.82:6789/0 conn(0x7f7b3c0cca20 legacy=0x7f7b3c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.769+0000 7f7b39e00640 1 -- 172.21.15.82:0/2113180774 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c15c930 con 0x7f7b3c159960 2024-03-16T17:58:23.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.769+0000 7f7b39400640 1 -- 172.21.15.82:0/2113180774 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442570349 0 0) 0x7f7b3c15c370 con 0x7f7b3c0cca20 2024-03-16T17:58:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.770+0000 7f7b39e00640 1 --2- 172.21.15.82:0/2113180774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 0x7f7b3c06e230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0097a0 tx=0x7f7b2c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae24dd9cb411e48e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.770+0000 7f7b39400640 1 -- 172.21.15.82:0/2113180774 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c02e070 con 0x7f7b3c159960 2024-03-16T17:58:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.770+0000 7f7b39400640 1 -- 172.21.15.82:0/2113180774 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c0315a0 con 0x7f7b3c159960 2024-03-16T17:58:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.770+0000 7f7b39400640 1 -- 172.21.15.82:0/2113180774 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c031930 con 0x7f7b3c159960 2024-03-16T17:58:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.771+0000 7f7b41200640 1 -- 172.21.15.82:0/2113180774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 msgr2=0x7f7b3c06e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.771+0000 7f7b41200640 1 --2- 172.21.15.82:0/2113180774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 0x7f7b3c06e230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0097a0 tx=0x7f7b2c02b840 comp rx=0 tx=0).stop 2024-03-16T17:58:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.771+0000 7f7b41200640 1 -- 172.21.15.82:0/2113180774 shutdown_connections 2024-03-16T17:58:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.771+0000 7f7b41200640 1 --2- 172.21.15.82:0/2113180774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 0x7f7b3c06e230 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.771+0000 7f7b41200640 1 -- 172.21.15.82:0/2113180774 >> 172.21.15.82:0/2113180774 conn(0x7f7b3c0b1640 msgr2=0x7f7b3c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.771+0000 7f7b41200640 1 -- 172.21.15.82:0/2113180774 shutdown_connections 2024-03-16T17:58:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.771+0000 7f7b41200640 1 -- 172.21.15.82:0/2113180774 wait complete. 2024-03-16T17:58:23.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.773+0000 7f7b41200640 1 Processor -- start 2024-03-16T17:58:23.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.773+0000 7f7b41200640 1 -- start start 2024-03-16T17:58:23.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.774+0000 7f7b41200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c06e770 0x7f7b3c136070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.774+0000 7f7b41200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3c0cca20 0x7f7b3c132d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.774+0000 7f7b41200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 0x7f7b3c15e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.774+0000 7f7b41200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c06ec20 con 0x7f7b3c159960 2024-03-16T17:58:23.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.774+0000 7f7b41200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c15cff0 con 0x7f7b3c06e770 2024-03-16T17:58:23.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.774+0000 7f7b41200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c159050 con 0x7f7b3c0cca20 2024-03-16T17:58:23.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.774+0000 7f7b3a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 0x7f7b3c15e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.774+0000 7f7b3a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 0x7f7b3c15e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47248/0 (socket says 172.21.15.82:47248) 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.774+0000 7f7b3a800640 1 -- 172.21.15.82:0/2326603231 learned_addr learned my addr 172.21.15.82:0/2326603231 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b39e00640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c06e770 0x7f7b3c136070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b33e00640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3c0cca20 0x7f7b3c132d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b3a800640 1 -- 172.21.15.82:0/2326603231 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3c0cca20 msgr2=0x7f7b3c132d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b3a800640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3c0cca20 0x7f7b3c132d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b3a800640 1 -- 172.21.15.82:0/2326603231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c06e770 msgr2=0x7f7b3c136070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b3a800640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c06e770 0x7f7b3c136070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b3a800640 1 -- 172.21.15.82:0/2326603231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c009400 con 0x7f7b3c159960 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b3a800640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 0x7f7b3c15e8d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400c7a0 tx=0x7f7b3400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b32000640 1 -- 172.21.15.82:0/2326603231 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3400a5e0 con 0x7f7b3c159960 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b32000640 1 -- 172.21.15.82:0/2326603231 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3400a740 con 0x7f7b3c159960 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.775+0000 7f7b41200640 1 -- 172.21.15.82:0/2326603231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c0b5660 con 0x7f7b3c159960 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.776+0000 7f7b32000640 1 -- 172.21.15.82:0/2326603231 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3400ab70 con 0x7f7b3c159960 2024-03-16T17:58:23.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.776+0000 7f7b41200640 1 -- 172.21.15.82:0/2326603231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3c133a40 con 0x7f7b3c159960 2024-03-16T17:58:23.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.777+0000 7f7b41200640 1 -- 172.21.15.82:0/2326603231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3c067f30 con 0x7f7b3c159960 2024-03-16T17:58:23.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.778+0000 7f7b32000640 1 -- 172.21.15.82:0/2326603231 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f7b3400acd0 con 0x7f7b3c159960 2024-03-16T17:58:23.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.779+0000 7f7b32000640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1806b3b0 0x7f7b1806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:23.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.779+0000 7f7b32000640 1 -- 172.21.15.82:0/2326603231 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b34089310 con 0x7f7b3c159960 2024-03-16T17:58:23.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.779+0000 7f7b39e00640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1806b3b0 0x7f7b1806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:23.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.780+0000 7f7b39e00640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1806b3b0 0x7f7b1806d870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c036000 tx=0x7f7b2c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:23.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.783+0000 7f7b32000640 1 -- 172.21.15.82:0/2326603231 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7b34054000 con 0x7f7b3c159960 2024-03-16T17:58:23.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.978+0000 7f7b41200640 1 -- 172.21.15.82:0/2326603231 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b3c0ceb80 con 0x7f7b3c159960 2024-03-16T17:58:23.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.979+0000 7f7b32000640 1 -- 172.21.15.82:0/2326603231 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b340576c0 con 0x7f7b3c159960 2024-03-16T17:58:23.983 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:23.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 -- 172.21.15.82:0/2326603231 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1806b3b0 msgr2=0x7f7b1806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1806b3b0 0x7f7b1806d870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c036000 tx=0x7f7b2c008030 comp rx=0 tx=0).stop 2024-03-16T17:58:23.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 -- 172.21.15.82:0/2326603231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 msgr2=0x7f7b3c15e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:23.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 0x7f7b3c15e8d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400c7a0 tx=0x7f7b3400cc60 comp rx=0 tx=0).stop 2024-03-16T17:58:23.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 -- 172.21.15.82:0/2326603231 shutdown_connections 2024-03-16T17:58:23.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7b1806b3b0 0x7f7b1806d870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b3c159960 0x7f7b3c15e8d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7b3c0cca20 0x7f7b3c132d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 --2- 172.21.15.82:0/2326603231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c06e770 0x7f7b3c136070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:23.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 -- 172.21.15.82:0/2326603231 >> 172.21.15.82:0/2326603231 conn(0x7f7b3c0b1640 msgr2=0x7f7b3c06dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:23.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 -- 172.21.15.82:0/2326603231 shutdown_connections 2024-03-16T17:58:23.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:23.982+0000 7f7b41200640 1 -- 172.21.15.82:0/2326603231 wait complete. 2024-03-16T17:58:24.004 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:58:24.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.215+0000 7f8592600640 1 Processor -- start 2024-03-16T17:58:24.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.215+0000 7f8592600640 1 -- start start 2024-03-16T17:58:24.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.217+0000 7f8592600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f858c0d0ba0 0x7f858c0d46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.217+0000 7f8592600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f858c0b28d0 0x7f858c0b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.217+0000 7f8592600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858c0d1790 con 0x7f858c15dd40 2024-03-16T17:58:24.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.217+0000 7f8592600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858c0d1a70 con 0x7f858c0b28d0 2024-03-16T17:58:24.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.217+0000 7f8592600640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858c0cf410 con 0x7f858c0d0ba0 2024-03-16T17:58:24.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.217+0000 7f8591200640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f858c15dd40 0x7f858c0d0490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47660/0 (socket says 172.21.15.82:47660) 2024-03-16T17:58:24.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.217+0000 7f8591200640 1 -- 172.21.15.82:0/3129513529 learned_addr learned my addr 172.21.15.82:0/3129513529 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:24.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.217+0000 7f8591c00640 1 --2- 172.21.15.82:0/3129513529 >> v2:172.21.15.82:3300/0 conn(0x7f858c0b28d0 0x7f858c0b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.217+0000 7f8583e00640 1 --2- 172.21.15.82:0/3129513529 >> v2:172.21.15.90:3300/0 conn(0x7f858c0d0ba0 0x7f858c0d46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.217+0000 7f858be00640 1 -- 172.21.15.82:0/3129513529 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248361311 0 0) 0x7f858c0d1790 con 0x7f858c15dd40 2024-03-16T17:58:24.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.218+0000 7f858be00640 1 -- 172.21.15.82:0/3129513529 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8570003520 con 0x7f858c15dd40 2024-03-16T17:58:24.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.218+0000 7f8591c00640 1 -- 172.21.15.82:0/3129513529 >> v2:172.21.15.90:3300/0 conn(0x7f858c0d0ba0 msgr2=0x7f858c0d46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:24.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.218+0000 7f8591c00640 1 --2- 172.21.15.82:0/3129513529 >> v2:172.21.15.90:3300/0 conn(0x7f858c0d0ba0 0x7f858c0d46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.218+0000 7f8591c00640 1 -- 172.21.15.82:0/3129513529 >> v1:172.21.15.82:6789/0 conn(0x7f858c15dd40 legacy=0x7f858c0d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.218+0000 7f8591c00640 1 -- 172.21.15.82:0/3129513529 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0cf6b0 con 0x7f858c0b28d0 2024-03-16T17:58:24.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.219+0000 7f8591c00640 1 --2- 172.21.15.82:0/3129513529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c0b28d0 0x7f858c0b07b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f858c08dba0 tx=0x7f858402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48538f00aa55e86e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:24.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.219+0000 7f858be00640 1 -- 172.21.15.82:0/3129513529 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858402cbe0 con 0x7f858c0b28d0 2024-03-16T17:58:24.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.219+0000 7f858be00640 1 -- 172.21.15.82:0/3129513529 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858402cd40 con 0x7f858c0b28d0 2024-03-16T17:58:24.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.219+0000 7f858be00640 1 -- 172.21.15.82:0/3129513529 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584034660 con 0x7f858c0b28d0 2024-03-16T17:58:24.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.220+0000 7f8592600640 1 -- 172.21.15.82:0/3129513529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c0b28d0 msgr2=0x7f858c0b07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.220+0000 7f8592600640 1 --2- 172.21.15.82:0/3129513529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c0b28d0 0x7f858c0b07b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f858c08dba0 tx=0x7f858402be30 comp rx=0 tx=0).stop 2024-03-16T17:58:24.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.220+0000 7f8592600640 1 -- 172.21.15.82:0/3129513529 shutdown_connections 2024-03-16T17:58:24.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.220+0000 7f8592600640 1 --2- 172.21.15.82:0/3129513529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c0b28d0 0x7f858c0b07b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.220+0000 7f8592600640 1 --2- 172.21.15.82:0/3129513529 >> v2:172.21.15.90:3300/0 conn(0x7f858c0d0ba0 0x7f858c0d46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.220+0000 7f8592600640 1 -- 172.21.15.82:0/3129513529 >> 172.21.15.82:0/3129513529 conn(0x7f858c0d3bc0 msgr2=0x7f858c0d3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:24.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.220+0000 7f8592600640 1 -- 172.21.15.82:0/3129513529 shutdown_connections 2024-03-16T17:58:24.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.220+0000 7f8592600640 1 -- 172.21.15.82:0/3129513529 wait complete. 2024-03-16T17:58:24.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.221+0000 7f8592600640 1 Processor -- start 2024-03-16T17:58:24.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.222+0000 7f8592600640 1 -- start start 2024-03-16T17:58:24.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.222+0000 7f8592600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f858c0b28d0 0x7f858c0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8592600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f858c0d0ba0 0x7f858c0a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8592600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c15dd40 0x7f858c154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8592600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c09dd90 con 0x7f858c15dd40 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8592600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c09dfa0 con 0x7f858c0d0ba0 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8592600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c09e280 con 0x7f858c0b28d0 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8591c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c15dd40 0x7f858c154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8591c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c15dd40 0x7f858c154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47276/0 (socket says 172.21.15.82:47276) 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8591c00640 1 -- 172.21.15.82:0/1924428123 learned_addr learned my addr 172.21.15.82:0/1924428123 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8591200640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f858c0b28d0 0x7f858c0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8583e00640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f858c0d0ba0 0x7f858c0a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8591c00640 1 -- 172.21.15.82:0/1924428123 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f858c0b28d0 msgr2=0x7f858c0cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8591c00640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f858c0b28d0 0x7f858c0cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8591c00640 1 -- 172.21.15.82:0/1924428123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f858c0d0ba0 msgr2=0x7f858c0a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8591c00640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f858c0d0ba0 0x7f858c0a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8591c00640 1 -- 172.21.15.82:0/1924428123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8584009a70 con 0x7f858c15dd40 2024-03-16T17:58:24.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.223+0000 7f8591200640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f858c0b28d0 0x7f858c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:24.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.224+0000 7f8591c00640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c15dd40 0x7f858c154fe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8584009e80 tx=0x7f8584004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:24.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.224+0000 7f8589600640 1 -- 172.21.15.82:0/1924428123 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584040650 con 0x7f858c15dd40 2024-03-16T17:58:24.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.224+0000 7f8589600640 1 -- 172.21.15.82:0/1924428123 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85840407b0 con 0x7f858c15dd40 2024-03-16T17:58:24.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.224+0000 7f8589600640 1 -- 172.21.15.82:0/1924428123 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584040b60 con 0x7f858c15dd40 2024-03-16T17:58:24.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.224+0000 7f8592600640 1 -- 172.21.15.82:0/1924428123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858c15e2a0 con 0x7f858c15dd40 2024-03-16T17:58:24.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.224+0000 7f8592600640 1 -- 172.21.15.82:0/1924428123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858c09e840 con 0x7f858c15dd40 2024-03-16T17:58:24.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.225+0000 7f8592600640 1 -- 172.21.15.82:0/1924428123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858c04f250 con 0x7f858c15dd40 2024-03-16T17:58:24.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.226+0000 7f8589600640 1 -- 172.21.15.82:0/1924428123 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8584040cc0 con 0x7f858c15dd40 2024-03-16T17:58:24.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.227+0000 7f8589600640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f856806b3b0 0x7f856806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.227+0000 7f8589600640 1 -- 172.21.15.82:0/1924428123 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85840ac920 con 0x7f858c15dd40 2024-03-16T17:58:24.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.227+0000 7f8591200640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f856806b3b0 0x7f856806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.228+0000 7f8591200640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f856806b3b0 0x7f856806d870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f857c005880 tx=0x7f857c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:24.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.231+0000 7f8589600640 1 -- 172.21.15.82:0/1924428123 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8584077730 con 0x7f858c15dd40 2024-03-16T17:58:24.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.430+0000 7f8592600640 1 -- 172.21.15.82:0/1924428123 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f858c09db70 con 0x7f858c15dd40 2024-03-16T17:58:24.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.431+0000 7f8589600640 1 -- 172.21.15.82:0/1924428123 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8584009450 con 0x7f858c15dd40 2024-03-16T17:58:24.435 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.433+0000 7f8592600640 1 -- 172.21.15.82:0/1924428123 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f856806b3b0 msgr2=0x7f856806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.433+0000 7f8592600640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f856806b3b0 0x7f856806d870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f857c005880 tx=0x7f857c009040 comp rx=0 tx=0).stop 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.433+0000 7f8592600640 1 -- 172.21.15.82:0/1924428123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c15dd40 msgr2=0x7f858c154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.433+0000 7f8592600640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c15dd40 0x7f858c154fe0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8584009e80 tx=0x7f8584004a40 comp rx=0 tx=0).stop 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.434+0000 7f8592600640 1 -- 172.21.15.82:0/1924428123 shutdown_connections 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.434+0000 7f8592600640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f856806b3b0 0x7f856806d870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.434+0000 7f8592600640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f858c15dd40 0x7f858c154fe0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.434+0000 7f8592600640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f858c0d0ba0 0x7f858c0a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.434+0000 7f8592600640 1 --2- 172.21.15.82:0/1924428123 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f858c0b28d0 0x7f858c0cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.434+0000 7f8592600640 1 -- 172.21.15.82:0/1924428123 >> 172.21.15.82:0/1924428123 conn(0x7f858c0d3bc0 msgr2=0x7f858c0d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.434+0000 7f8592600640 1 -- 172.21.15.82:0/1924428123 shutdown_connections 2024-03-16T17:58:24.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.434+0000 7f8592600640 1 -- 172.21.15.82:0/1924428123 wait complete. 2024-03-16T17:58:24.456 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:58:24.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.653+0000 7f5a32c00640 1 Processor -- start 2024-03-16T17:58:24.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.654+0000 7f5a32c00640 1 -- start start 2024-03-16T17:58:24.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.655+0000 7f5a32c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c14b770 0x7f5a2c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.655+0000 7f5a32c00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5a2c0cf130 0x7f5a2c0cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.655+0000 7f5a32c00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2c0cac70 con 0x7f5a2c0cc8c0 2024-03-16T17:58:24.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.655+0000 7f5a32c00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2c0ccd50 con 0x7f5a2c0cf130 2024-03-16T17:58:24.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.655+0000 7f5a32c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c14bb50 con 0x7f5a2c14b770 2024-03-16T17:58:24.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.656+0000 7f5a23400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5a2c0cf130 0x7f5a2c0cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.656+0000 7f5a23400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5a2c0cf130 0x7f5a2c0cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:38402/0 (socket says 172.21.15.82:38402) 2024-03-16T17:58:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.656+0000 7f5a23400640 1 -- 172.21.15.82:0/2359689353 learned_addr learned my addr 172.21.15.82:0/2359689353 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.656+0000 7f5a2be00640 1 --2- 172.21.15.82:0/2359689353 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c14b770 0x7f5a2c06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.657+0000 7f5a23400640 1 -- 172.21.15.82:0/2359689353 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c14b770 msgr2=0x7f5a2c06cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.657+0000 7f5a23400640 1 --2- 172.21.15.82:0/2359689353 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c14b770 0x7f5a2c06cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.657+0000 7f5a23400640 1 -- 172.21.15.82:0/2359689353 >> v1:172.21.15.90:6789/0 conn(0x7f5a2c0cc8c0 legacy=0x7f5a2c165f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.657+0000 7f5a23400640 1 -- 172.21.15.82:0/2359689353 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c0ca060 con 0x7f5a2c0cf130 2024-03-16T17:58:24.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.658+0000 7f5a23400640 1 --2- 172.21.15.82:0/2359689353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0cf130 0x7f5a2c0cc380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5a140098a0 tx=0x7f5a1402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183b29dfc7881542 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:24.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.658+0000 7f5a2b400640 1 -- 172.21.15.82:0/2359689353 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14030840 con 0x7f5a2c0cf130 2024-03-16T17:58:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.658+0000 7f5a2b400640 1 -- 172.21.15.82:0/2359689353 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a140309a0 con 0x7f5a2c0cf130 2024-03-16T17:58:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.658+0000 7f5a2b400640 1 -- 172.21.15.82:0/2359689353 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14030d30 con 0x7f5a2c0cf130 2024-03-16T17:58:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.659+0000 7f5a32c00640 1 -- 172.21.15.82:0/2359689353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0cf130 msgr2=0x7f5a2c0cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.659+0000 7f5a32c00640 1 --2- 172.21.15.82:0/2359689353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0cf130 0x7f5a2c0cc380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5a140098a0 tx=0x7f5a1402b980 comp rx=0 tx=0).stop 2024-03-16T17:58:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.659+0000 7f5a32c00640 1 -- 172.21.15.82:0/2359689353 shutdown_connections 2024-03-16T17:58:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.659+0000 7f5a32c00640 1 --2- 172.21.15.82:0/2359689353 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0cf130 0x7f5a2c0cc380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.659+0000 7f5a32c00640 1 --2- 172.21.15.82:0/2359689353 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c14b770 0x7f5a2c06cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.659+0000 7f5a32c00640 1 -- 172.21.15.82:0/2359689353 >> 172.21.15.82:0/2359689353 conn(0x7f5a2c0b1640 msgr2=0x7f5a2c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.659+0000 7f5a32c00640 1 -- 172.21.15.82:0/2359689353 shutdown_connections 2024-03-16T17:58:24.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.659+0000 7f5a32c00640 1 -- 172.21.15.82:0/2359689353 wait complete. 2024-03-16T17:58:24.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.660+0000 7f5a32c00640 1 Processor -- start 2024-03-16T17:58:24.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.661+0000 7f5a32c00640 1 -- start start 2024-03-16T17:58:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.661+0000 7f5a32c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0cc8c0 0x7f5a2c161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.661+0000 7f5a32c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c0cf130 0x7f5a2c0c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.661+0000 7f5a32c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c14b770 0x7f5a2c0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.661+0000 7f5a32c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c0c6110 con 0x7f5a2c0cf130 2024-03-16T17:58:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.661+0000 7f5a32c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c0c5e30 con 0x7f5a2c0cc8c0 2024-03-16T17:58:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.661+0000 7f5a32c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c149fc0 con 0x7f5a2c14b770 2024-03-16T17:58:24.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.662+0000 7f5a23400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c0cf130 0x7f5a2c0c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.662+0000 7f5a23400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c0cf130 0x7f5a2c0c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47282/0 (socket says 172.21.15.82:47282) 2024-03-16T17:58:24.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.662+0000 7f5a23400640 1 -- 172.21.15.82:0/3053051296 learned_addr learned my addr 172.21.15.82:0/3053051296 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.662+0000 7f5a2be00640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0cc8c0 0x7f5a2c161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.662+0000 7f5a23400640 1 -- 172.21.15.82:0/3053051296 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c14b770 msgr2=0x7f5a2c0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.662+0000 7f5a23400640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c14b770 0x7f5a2c0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.662+0000 7f5a23400640 1 -- 172.21.15.82:0/3053051296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0cc8c0 msgr2=0x7f5a2c161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.662+0000 7f5a23400640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0cc8c0 0x7f5a2c161e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.662+0000 7f5a23400640 1 -- 172.21.15.82:0/3053051296 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140094d0 con 0x7f5a2c0cf130 2024-03-16T17:58:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.662+0000 7f5a23400640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c0cf130 0x7f5a2c0c54e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5a140098a0 tx=0x7f5a14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.663+0000 7f5a28c00640 1 -- 172.21.15.82:0/3053051296 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14039c90 con 0x7f5a2c0cf130 2024-03-16T17:58:24.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.663+0000 7f5a28c00640 1 -- 172.21.15.82:0/3053051296 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a14039e10 con 0x7f5a2c0cf130 2024-03-16T17:58:24.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.663+0000 7f5a32c00640 1 -- 172.21.15.82:0/3053051296 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2c14a2a0 con 0x7f5a2c0cf130 2024-03-16T17:58:24.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.663+0000 7f5a28c00640 1 -- 172.21.15.82:0/3053051296 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14043640 con 0x7f5a2c0cf130 2024-03-16T17:58:24.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.663+0000 7f5a32c00640 1 -- 172.21.15.82:0/3053051296 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c0c5a20 con 0x7f5a2c0cf130 2024-03-16T17:58:24.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.664+0000 7f5a32c00640 1 -- 172.21.15.82:0/3053051296 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2c14a7c0 con 0x7f5a2c0cf130 2024-03-16T17:58:24.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.665+0000 7f5a28c00640 1 -- 172.21.15.82:0/3053051296 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f5a140437a0 con 0x7f5a2c0cf130 2024-03-16T17:58:24.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.666+0000 7f5a28c00640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5a0806b3b0 0x7f5a0806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:24.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.666+0000 7f5a28c00640 1 -- 172.21.15.82:0/3053051296 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a1403e070 con 0x7f5a2c0cf130 2024-03-16T17:58:24.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.666+0000 7f5a2be00640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5a0806b3b0 0x7f5a0806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:24.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.667+0000 7f5a2be00640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5a0806b3b0 0x7f5a0806d870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009a10 tx=0x7f5a1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:24.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.671+0000 7f5a28c00640 1 -- 172.21.15.82:0/3053051296 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5a2c14a7c0 con 0x7f5a2c0cf130 2024-03-16T17:58:24.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.864+0000 7f5a32c00640 1 -- 172.21.15.82:0/3053051296 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a2c14a7c0 con 0x7f5a2c0cf130 2024-03-16T17:58:24.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.865+0000 7f5a28c00640 1 -- 172.21.15.82:0/3053051296 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a2c14a7c0 con 0x7f5a2c0cf130 2024-03-16T17:58:24.868 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:58:24.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.867+0000 7f5a32c00640 1 -- 172.21.15.82:0/3053051296 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5a0806b3b0 msgr2=0x7f5a0806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.867+0000 7f5a32c00640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5a0806b3b0 0x7f5a0806d870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009a10 tx=0x7f5a1c008040 comp rx=0 tx=0).stop 2024-03-16T17:58:24.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.867+0000 7f5a32c00640 1 -- 172.21.15.82:0/3053051296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c0cf130 msgr2=0x7f5a2c0c54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:24.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.867+0000 7f5a32c00640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c0cf130 0x7f5a2c0c54e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5a140098a0 tx=0x7f5a14004730 comp rx=0 tx=0).stop 2024-03-16T17:58:24.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.867+0000 7f5a32c00640 1 -- 172.21.15.82:0/3053051296 shutdown_connections 2024-03-16T17:58:24.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.867+0000 7f5a32c00640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5a0806b3b0 0x7f5a0806d870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.867+0000 7f5a32c00640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5a2c14b770 0x7f5a2c0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.867+0000 7f5a32c00640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a2c0cf130 0x7f5a2c0c54e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.867+0000 7f5a32c00640 1 --2- 172.21.15.82:0/3053051296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a2c0cc8c0 0x7f5a2c161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:24.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.867+0000 7f5a32c00640 1 -- 172.21.15.82:0/3053051296 >> 172.21.15.82:0/3053051296 conn(0x7f5a2c0b1640 msgr2=0x7f5a2c071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:24.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.868+0000 7f5a32c00640 1 -- 172.21.15.82:0/3053051296 shutdown_connections 2024-03-16T17:58:24.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:24.868+0000 7f5a32c00640 1 -- 172.21.15.82:0/3053051296 wait complete. 2024-03-16T17:58:24.890 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T17:58:25.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.098+0000 7f7900200640 1 Processor -- start 2024-03-16T17:58:25.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.098+0000 7f7900200640 1 -- start start 2024-03-16T17:58:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f7900200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f78f806d410 0x7f78f815f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f7900200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f80cddd0 con 0x7f78f806c870 2024-03-16T17:58:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f7900200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f815bf80 con 0x7f78f80ce820 2024-03-16T17:58:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f7900200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f815f570 con 0x7f78f806d410 2024-03-16T17:58:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f78fd800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f78f806c870 0x7f78f806cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39898/0 (socket says 172.21.15.82:39898) 2024-03-16T17:58:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f78fd800640 1 -- 172.21.15.82:0/1326529480 learned_addr learned my addr 172.21.15.82:0/1326529480 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:25.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f78fce00640 1 --2- 172.21.15.82:0/1326529480 >> v2:172.21.15.90:3300/0 conn(0x7f78f806d410 0x7f78f815f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f78f7e00640 1 -- 172.21.15.82:0/1326529480 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126484329 0 0) 0x7f78f80cddd0 con 0x7f78f806c870 2024-03-16T17:58:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f78f7e00640 1 -- 172.21.15.82:0/1326529480 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e0003520 con 0x7f78f806c870 2024-03-16T17:58:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f78f7e00640 1 -- 172.21.15.82:0/1326529480 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 108376778 0 0) 0x7f78e0003520 con 0x7f78f806c870 2024-03-16T17:58:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f78fce00640 1 -- 172.21.15.82:0/1326529480 >> v1:172.21.15.90:6789/0 conn(0x7f78f80ce820 legacy=0x7f78f80b2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f78fce00640 1 -- 172.21.15.82:0/1326529480 >> v1:172.21.15.82:6789/0 conn(0x7f78f806c870 legacy=0x7f78f806cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.100+0000 7f78fce00640 1 -- 172.21.15.82:0/1326529480 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80b4690 con 0x7f78f806d410 2024-03-16T17:58:25.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.102+0000 7f78fce00640 1 --2- 172.21.15.82:0/1326529480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f806d410 0x7f78f815f030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f78e80097a0 tx=0x7f78e802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa835d3aa2b1f728 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.103+0000 7f78f7e00640 1 -- 172.21.15.82:0/1326529480 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e802f050 con 0x7f78f806d410 2024-03-16T17:58:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.103+0000 7f78f7e00640 1 -- 172.21.15.82:0/1326529480 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e8031500 con 0x7f78f806d410 2024-03-16T17:58:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.103+0000 7f78f7e00640 1 -- 172.21.15.82:0/1326529480 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec001670 con 0x7f78f806d410 2024-03-16T17:58:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.103+0000 7f7900200640 1 -- 172.21.15.82:0/1326529480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f806d410 msgr2=0x7f78f815f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.103+0000 7f7900200640 1 --2- 172.21.15.82:0/1326529480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f806d410 0x7f78f815f030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f78e80097a0 tx=0x7f78e802b820 comp rx=0 tx=0).stop 2024-03-16T17:58:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.103+0000 7f7900200640 1 -- 172.21.15.82:0/1326529480 shutdown_connections 2024-03-16T17:58:25.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.103+0000 7f7900200640 1 --2- 172.21.15.82:0/1326529480 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f806d410 0x7f78f815f030 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.104+0000 7f7900200640 1 -- 172.21.15.82:0/1326529480 >> 172.21.15.82:0/1326529480 conn(0x7f78f80b17b0 msgr2=0x7f78f80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:25.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.104+0000 7f7900200640 1 -- 172.21.15.82:0/1326529480 shutdown_connections 2024-03-16T17:58:25.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.104+0000 7f7900200640 1 -- 172.21.15.82:0/1326529480 wait complete. 2024-03-16T17:58:25.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.105+0000 7f7900200640 1 Processor -- start 2024-03-16T17:58:25.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.105+0000 7f7900200640 1 -- start start 2024-03-16T17:58:25.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.106+0000 7f7900200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f806c870 0x7f78f81f7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.106+0000 7f7900200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f806d410 0x7f78f81500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.106+0000 7f7900200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f80ce820 0x7f78f80b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.106+0000 7f7900200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80afa30 con 0x7f78f80ce820 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.106+0000 7f7900200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f81f80c0 con 0x7f78f806d410 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.106+0000 7f7900200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f8157180 con 0x7f78f806c870 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.107+0000 7f78fe200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f80ce820 0x7f78f80b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.107+0000 7f78fe200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f80ce820 0x7f78f80b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58660/0 (socket says 172.21.15.82:58660) 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.107+0000 7f78fe200640 1 -- 172.21.15.82:0/1343034650 learned_addr learned my addr 172.21.15.82:0/1343034650 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.107+0000 7f78fce00640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f806d410 0x7f78f81500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.107+0000 7f78fd800640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f806c870 0x7f78f81f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.107+0000 7f78fe200640 1 -- 172.21.15.82:0/1343034650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f806c870 msgr2=0x7f78f81f7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.107+0000 7f78fe200640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f806c870 0x7f78f81f7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.107+0000 7f78fe200640 1 -- 172.21.15.82:0/1343034650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f806d410 msgr2=0x7f78f81500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.107+0000 7f78fe200640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f806d410 0x7f78f81500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.107+0000 7f78fe200640 1 -- 172.21.15.82:0/1343034650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8009400 con 0x7f78f80ce820 2024-03-16T17:58:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.108+0000 7f78fe200640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f80ce820 0x7f78f80b2d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c7a0 tx=0x7f78f000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.108+0000 7f78f5600640 1 -- 172.21.15.82:0/1343034650 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0003b70 con 0x7f78f80ce820 2024-03-16T17:58:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.108+0000 7f78f5600640 1 -- 172.21.15.82:0/1343034650 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f0003cf0 con 0x7f78f80ce820 2024-03-16T17:58:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.108+0000 7f78f5600640 1 -- 172.21.15.82:0/1343034650 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0003ea0 con 0x7f78f80ce820 2024-03-16T17:58:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.108+0000 7f7900200640 1 -- 172.21.15.82:0/1343034650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f81f7820 con 0x7f78f80ce820 2024-03-16T17:58:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.108+0000 7f7900200640 1 -- 172.21.15.82:0/1343034650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f81f7de0 con 0x7f78f80ce820 2024-03-16T17:58:25.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.109+0000 7f7900200640 1 -- 172.21.15.82:0/1343034650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f80afba0 con 0x7f78f80ce820 2024-03-16T17:58:25.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.110+0000 7f78f5600640 1 -- 172.21.15.82:0/1343034650 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f78f000aa90 con 0x7f78f80ce820 2024-03-16T17:58:25.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.111+0000 7f78f5600640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78dc06b3b0 0x7f78dc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.111+0000 7f78f5600640 1 -- 172.21.15.82:0/1343034650 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78f0089b20 con 0x7f78f80ce820 2024-03-16T17:58:25.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.111+0000 7f78fd800640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78dc06b3b0 0x7f78dc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.112+0000 7f78fd800640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78dc06b3b0 0x7f78dc06d870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0011e0 tx=0x7f78ec009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:25.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.115+0000 7f78f5600640 1 -- 172.21.15.82:0/1343034650 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f78f008be60 con 0x7f78f80ce820 2024-03-16T17:58:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.322+0000 7f7900200640 1 -- 172.21.15.82:0/1343034650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f78f8073420 con 0x7f78f80ce820 2024-03-16T17:58:25.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.323+0000 7f78f5600640 1 -- 172.21.15.82:0/1343034650 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f78f00546e0 con 0x7f78f80ce820 2024-03-16T17:58:25.327 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-16T17:58:25.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 -- 172.21.15.82:0/1343034650 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78dc06b3b0 msgr2=0x7f78dc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78dc06b3b0 0x7f78dc06d870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f78ec0011e0 tx=0x7f78ec009040 comp rx=0 tx=0).stop 2024-03-16T17:58:25.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 -- 172.21.15.82:0/1343034650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f80ce820 msgr2=0x7f78f80b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f80ce820 0x7f78f80b2d50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f78f000c7a0 tx=0x7f78f000cc60 comp rx=0 tx=0).stop 2024-03-16T17:58:25.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 -- 172.21.15.82:0/1343034650 shutdown_connections 2024-03-16T17:58:25.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78dc06b3b0 0x7f78dc06d870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f80ce820 0x7f78f80b2d50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f806d410 0x7f78f81500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 --2- 172.21.15.82:0/1343034650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f806c870 0x7f78f81f7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 -- 172.21.15.82:0/1343034650 >> 172.21.15.82:0/1343034650 conn(0x7f78f80b17b0 msgr2=0x7f78f80cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:25.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 -- 172.21.15.82:0/1343034650 shutdown_connections 2024-03-16T17:58:25.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.326+0000 7f7900200640 1 -- 172.21.15.82:0/1343034650 wait complete. 2024-03-16T17:58:25.348 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-16T17:58:25.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:25.348 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-16T17:58:25.443 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T17:58:25.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:25.443 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-16T17:58:25.499 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:58:25.499 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:25.553 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:58:25.553 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:25.646 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:58:25.646 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:25.698 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:25.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.911+0000 7f3436a00640 1 Processor -- start 2024-03-16T17:58:25.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.911+0000 7f3436a00640 1 -- start start 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.912+0000 7f3436a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f34300d00a0 0x7f343015b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.912+0000 7f3436a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f343015f550 0x7f343015b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.912+0000 7f3436a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343015e820 con 0x7f3430144400 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.912+0000 7f3436a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343015f130 con 0x7f343015f550 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.912+0000 7f3436a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343015c080 con 0x7f34300d00a0 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.913+0000 7f342fe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3430144400 0x7f34300cf990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39904/0 (socket says 172.21.15.82:39904) 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.913+0000 7f342fe00640 1 -- 172.21.15.82:0/2103669078 learned_addr learned my addr 172.21.15.82:0/2103669078 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.913+0000 7f3434a00640 1 --2- 172.21.15.82:0/2103669078 >> v2:172.21.15.82:3300/0 conn(0x7f343015f550 0x7f343015b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.913+0000 7f3427400640 1 --2- 172.21.15.82:0/2103669078 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f34300d00a0 0x7f343015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.913+0000 7f342f400640 1 -- 172.21.15.82:0/2103669078 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832587729 0 0) 0x7f343015e820 con 0x7f3430144400 2024-03-16T17:58:25.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.913+0000 7f342f400640 1 -- 172.21.15.82:0/2103669078 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3414003520 con 0x7f3430144400 2024-03-16T17:58:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.913+0000 7f3434a00640 1 -- 172.21.15.82:0/2103669078 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f34300d00a0 msgr2=0x7f343015b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.913+0000 7f3434a00640 1 --2- 172.21.15.82:0/2103669078 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f34300d00a0 0x7f343015b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.913+0000 7f3434a00640 1 -- 172.21.15.82:0/2103669078 >> v1:172.21.15.82:6789/0 conn(0x7f3430144400 legacy=0x7f34300cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.913+0000 7f3434a00640 1 -- 172.21.15.82:0/2103669078 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343015ea70 con 0x7f343015f550 2024-03-16T17:58:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.914+0000 7f3434a00640 1 --2- 172.21.15.82:0/2103669078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343015f550 0x7f343015b850 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3428009d90 tx=0x7f342802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1ba9f9ddfdf47 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:25.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.914+0000 7f342f400640 1 -- 172.21.15.82:0/2103669078 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342802cbc0 con 0x7f343015f550 2024-03-16T17:58:25.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.914+0000 7f342f400640 1 -- 172.21.15.82:0/2103669078 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342802cd20 con 0x7f343015f550 2024-03-16T17:58:25.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.914+0000 7f342f400640 1 -- 172.21.15.82:0/2103669078 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428034660 con 0x7f343015f550 2024-03-16T17:58:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.916+0000 7f3436a00640 1 -- 172.21.15.82:0/2103669078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343015f550 msgr2=0x7f343015b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.916+0000 7f3436a00640 1 --2- 172.21.15.82:0/2103669078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343015f550 0x7f343015b850 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3428009d90 tx=0x7f342802be10 comp rx=0 tx=0).stop 2024-03-16T17:58:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.916+0000 7f3436a00640 1 -- 172.21.15.82:0/2103669078 shutdown_connections 2024-03-16T17:58:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.916+0000 7f3436a00640 1 --2- 172.21.15.82:0/2103669078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f343015f550 0x7f343015b850 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.916+0000 7f3436a00640 1 --2- 172.21.15.82:0/2103669078 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f34300d00a0 0x7f343015b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.916+0000 7f3436a00640 1 -- 172.21.15.82:0/2103669078 >> 172.21.15.82:0/2103669078 conn(0x7f343006ae70 msgr2=0x7f343006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.916+0000 7f3436a00640 1 -- 172.21.15.82:0/2103669078 shutdown_connections 2024-03-16T17:58:25.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.916+0000 7f3436a00640 1 -- 172.21.15.82:0/2103669078 wait complete. 2024-03-16T17:58:25.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.918+0000 7f3436a00640 1 Processor -- start 2024-03-16T17:58:25.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.918+0000 7f3436a00640 1 -- start start 2024-03-16T17:58:25.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f3436a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34300d00a0 0x7f343013dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f3436a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3430144400 0x7f34301f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f3436a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f343015f550 0x7f34300b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f3436a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343013ced0 con 0x7f34300d00a0 2024-03-16T17:58:25.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f3436a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34301f4680 con 0x7f3430144400 2024-03-16T17:58:25.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f3436a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343013d170 con 0x7f343015f550 2024-03-16T17:58:25.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f342fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34300d00a0 0x7f343013dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f342fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34300d00a0 0x7f343013dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58672/0 (socket says 172.21.15.82:58672) 2024-03-16T17:58:25.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f342fe00640 1 -- 172.21.15.82:0/1922951298 learned_addr learned my addr 172.21.15.82:0/1922951298 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f342fe00640 1 -- 172.21.15.82:0/1922951298 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f343015f550 msgr2=0x7f34300b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f3434a00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f343015f550 0x7f34300b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f3427400640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3430144400 0x7f34301f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f342fe00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f343015f550 0x7f34300b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f342fe00640 1 -- 172.21.15.82:0/1922951298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3430144400 msgr2=0x7f34301f0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f342fe00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3430144400 0x7f34301f0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.919+0000 7f342fe00640 1 -- 172.21.15.82:0/1922951298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3428009a70 con 0x7f34300d00a0 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.920+0000 7f342fe00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34300d00a0 0x7f343013dc50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f341800c390 tx=0x7f341800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.920+0000 7f342cc00640 1 -- 172.21.15.82:0/1922951298 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418005ce0 con 0x7f34300d00a0 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.920+0000 7f342cc00640 1 -- 172.21.15.82:0/1922951298 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3418005e40 con 0x7f34300d00a0 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.920+0000 7f3436a00640 1 -- 172.21.15.82:0/1922951298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343013d450 con 0x7f34300d00a0 2024-03-16T17:58:25.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.920+0000 7f342cc00640 1 -- 172.21.15.82:0/1922951298 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34180072f0 con 0x7f34300d00a0 2024-03-16T17:58:25.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.920+0000 7f3436a00640 1 -- 172.21.15.82:0/1922951298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343013e190 con 0x7f34300d00a0 2024-03-16T17:58:25.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.921+0000 7f3436a00640 1 -- 172.21.15.82:0/1922951298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343013d970 con 0x7f34300d00a0 2024-03-16T17:58:25.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.922+0000 7f342cc00640 1 -- 172.21.15.82:0/1922951298 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f3418007450 con 0x7f34300d00a0 2024-03-16T17:58:25.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.923+0000 7f342cc00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f340c06b3b0 0x7f340c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:25.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.923+0000 7f342cc00640 1 -- 172.21.15.82:0/1922951298 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34180892c0 con 0x7f34300d00a0 2024-03-16T17:58:25.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.923+0000 7f3427400640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f340c06b3b0 0x7f340c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:25.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.924+0000 7f3427400640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f340c06b3b0 0x7f340c06d870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3420006fd0 tx=0x7f3420008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:25.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:25.928+0000 7f342cc00640 1 -- 172.21.15.82:0/1922951298 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f343013d970 con 0x7f34300d00a0 2024-03-16T17:58:26.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.124+0000 7f3436a00640 1 -- 172.21.15.82:0/1922951298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f343013d970 con 0x7f34300d00a0 2024-03-16T17:58:26.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.126+0000 7f342cc00640 1 -- 172.21.15.82:0/1922951298 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f343013d970 con 0x7f34300d00a0 2024-03-16T17:58:26.129 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:26.129 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":21,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:57:42.360239+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 -- 172.21.15.82:0/1922951298 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f340c06b3b0 msgr2=0x7f340c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f340c06b3b0 0x7f340c06d870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3420006fd0 tx=0x7f3420008040 comp rx=0 tx=0).stop 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 -- 172.21.15.82:0/1922951298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34300d00a0 msgr2=0x7f343013dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34300d00a0 0x7f343013dc50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f341800c390 tx=0x7f341800c850 comp rx=0 tx=0).stop 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 -- 172.21.15.82:0/1922951298 shutdown_connections 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f340c06b3b0 0x7f340c06d870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f343015f550 0x7f34300b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3430144400 0x7f34301f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 --2- 172.21.15.82:0/1922951298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34300d00a0 0x7f343013dc50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 -- 172.21.15.82:0/1922951298 >> 172.21.15.82:0/1922951298 conn(0x7f343006ae70 msgr2=0x7f3430143610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:26.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 -- 172.21.15.82:0/1922951298 shutdown_connections 2024-03-16T17:58:26.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.129+0000 7f3436a00640 1 -- 172.21.15.82:0/1922951298 wait complete. 2024-03-16T17:58:26.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:26.155 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:26.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.369+0000 7f76cb600640 1 Processor -- start 2024-03-16T17:58:26.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.369+0000 7f76cb600640 1 -- start start 2024-03-16T17:58:26.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.370+0000 7f76cb600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f76c4144330 0x7f76c4070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.370+0000 7f76cb600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c40714f0 0x7f76c415e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.370+0000 7f76cb600640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c40ce8f0 con 0x7f76c415eca0 2024-03-16T17:58:26.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.370+0000 7f76cb600640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c415f2d0 con 0x7f76c4144330 2024-03-16T17:58:26.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.370+0000 7f76cb600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c415f5b0 con 0x7f76c40714f0 2024-03-16T17:58:26.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.371+0000 7f76c8c00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f76c4144330 0x7f76c4070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.371+0000 7f76c8c00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f76c4144330 0x7f76c4070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:48870/0 (socket says 172.21.15.82:48870) 2024-03-16T17:58:26.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.371+0000 7f76c8c00640 1 -- 172.21.15.82:0/3685644894 learned_addr learned my addr 172.21.15.82:0/3685644894 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:26.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.371+0000 7f76bbe00640 1 --2- 172.21.15.82:0/3685644894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c40714f0 0x7f76c415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.371+0000 7f76c8c00640 1 -- 172.21.15.82:0/3685644894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c40714f0 msgr2=0x7f76c415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:26.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.371+0000 7f76c8c00640 1 --2- 172.21.15.82:0/3685644894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c40714f0 0x7f76c415e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.371+0000 7f76c8c00640 1 -- 172.21.15.82:0/3685644894 >> v1:172.21.15.90:6789/0 conn(0x7f76c415eca0 legacy=0x7f76c40cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.371+0000 7f76c8c00640 1 -- 172.21.15.82:0/3685644894 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40cacb0 con 0x7f76c4144330 2024-03-16T17:58:26.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.373+0000 7f76c8c00640 1 --2- 172.21.15.82:0/3685644894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c4144330 0x7f76c4070fb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009870 tx=0x7f76b402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a239726a4a2b0ae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:26.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.373+0000 7f76c3e00640 1 -- 172.21.15.82:0/3685644894 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4030840 con 0x7f76c4144330 2024-03-16T17:58:26.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.373+0000 7f76c3e00640 1 -- 172.21.15.82:0/3685644894 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b40309a0 con 0x7f76c4144330 2024-03-16T17:58:26.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.373+0000 7f76c3e00640 1 -- 172.21.15.82:0/3685644894 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4030d30 con 0x7f76c4144330 2024-03-16T17:58:26.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.374+0000 7f76cb600640 1 -- 172.21.15.82:0/3685644894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c4144330 msgr2=0x7f76c4070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.374+0000 7f76cb600640 1 --2- 172.21.15.82:0/3685644894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c4144330 0x7f76c4070fb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009870 tx=0x7f76b402b980 comp rx=0 tx=0).stop 2024-03-16T17:58:26.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.374+0000 7f76cb600640 1 -- 172.21.15.82:0/3685644894 shutdown_connections 2024-03-16T17:58:26.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.374+0000 7f76cb600640 1 --2- 172.21.15.82:0/3685644894 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c40714f0 0x7f76c415e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.374+0000 7f76cb600640 1 --2- 172.21.15.82:0/3685644894 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c4144330 0x7f76c4070fb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.374+0000 7f76cb600640 1 -- 172.21.15.82:0/3685644894 >> 172.21.15.82:0/3685644894 conn(0x7f76c40b1640 msgr2=0x7f76c40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:26.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.374+0000 7f76cb600640 1 -- 172.21.15.82:0/3685644894 shutdown_connections 2024-03-16T17:58:26.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.374+0000 7f76cb600640 1 -- 172.21.15.82:0/3685644894 wait complete. 2024-03-16T17:58:26.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.375+0000 7f76cb600640 1 Processor -- start 2024-03-16T17:58:26.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.376+0000 7f76cb600640 1 -- start start 2024-03-16T17:58:26.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76cb600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c40714f0 0x7f76c40ab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76cb600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76c4144330 0x7f76c40a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76cb600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c415eca0 0x7f76c41f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76cb600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40aa910 con 0x7f76c4144330 2024-03-16T17:58:26.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76cb600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40aa390 con 0x7f76c40714f0 2024-03-16T17:58:26.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76cb600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c41f7070 con 0x7f76c415eca0 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76bbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76c4144330 0x7f76c40a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76bbe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76c4144330 0x7f76c40a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58676/0 (socket says 172.21.15.82:58676) 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76bbe00640 1 -- 172.21.15.82:0/1717784935 learned_addr learned my addr 172.21.15.82:0/1717784935 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76c8c00640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c40714f0 0x7f76c40ab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76c9600640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c415eca0 0x7f76c41f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76bbe00640 1 -- 172.21.15.82:0/1717784935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c415eca0 msgr2=0x7f76c41f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76bbe00640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c415eca0 0x7f76c41f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76bbe00640 1 -- 172.21.15.82:0/1717784935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c40714f0 msgr2=0x7f76c40ab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76bbe00640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c40714f0 0x7f76c40ab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76bbe00640 1 -- 172.21.15.82:0/1717784935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b40094d0 con 0x7f76c4144330 2024-03-16T17:58:26.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.377+0000 7f76c9600640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c415eca0 0x7f76c41f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:26.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.378+0000 7f76bbe00640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76c4144330 0x7f76c40a9e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00e9b0 tx=0x7f76ac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:26.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.378+0000 7f76c1600640 1 -- 172.21.15.82:0/1717784935 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac009670 con 0x7f76c4144330 2024-03-16T17:58:26.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.378+0000 7f76c1600640 1 -- 172.21.15.82:0/1717784935 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac00c9a0 con 0x7f76c4144330 2024-03-16T17:58:26.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.378+0000 7f76cb600640 1 -- 172.21.15.82:0/1717784935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c40aa630 con 0x7f76c4144330 2024-03-16T17:58:26.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.378+0000 7f76c1600640 1 -- 172.21.15.82:0/1717784935 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00cd50 con 0x7f76c4144330 2024-03-16T17:58:26.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.378+0000 7f76cb600640 1 -- 172.21.15.82:0/1717784935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c40b2b40 con 0x7f76c4144330 2024-03-16T17:58:26.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.379+0000 7f76cb600640 1 -- 172.21.15.82:0/1717784935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c4067f30 con 0x7f76c4144330 2024-03-16T17:58:26.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.381+0000 7f76c1600640 1 -- 172.21.15.82:0/1717784935 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f76ac013470 con 0x7f76c4144330 2024-03-16T17:58:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.382+0000 7f76c1600640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f76a006b3b0 0x7f76a006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.382+0000 7f76c1600640 1 -- 172.21.15.82:0/1717784935 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f76ac089e50 con 0x7f76c4144330 2024-03-16T17:58:26.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.382+0000 7f76c8c00640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f76a006b3b0 0x7f76a006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.383+0000 7f76c8c00640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f76a006b3b0 0x7f76a006d870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76b4036000 tx=0x7f76b4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:26.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.386+0000 7f76c1600640 1 -- 172.21.15.82:0/1717784935 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f76ac054ab0 con 0x7f76c4144330 2024-03-16T17:58:26.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.582+0000 7f76cb600640 1 -- 172.21.15.82:0/1717784935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76c40ca540 con 0x7f76c4144330 2024-03-16T17:58:26.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.583+0000 7f76c1600640 1 -- 172.21.15.82:0/1717784935 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76ac0170a0 con 0x7f76c4144330 2024-03-16T17:58:26.586 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 -- 172.21.15.82:0/1717784935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f76a006b3b0 msgr2=0x7f76a006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f76a006b3b0 0x7f76a006d870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76b4036000 tx=0x7f76b4008030 comp rx=0 tx=0).stop 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 -- 172.21.15.82:0/1717784935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76c4144330 msgr2=0x7f76c40a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76c4144330 0x7f76c40a9e50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00e9b0 tx=0x7f76ac00ee70 comp rx=0 tx=0).stop 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 -- 172.21.15.82:0/1717784935 shutdown_connections 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f76a006b3b0 0x7f76a006d870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76c415eca0 0x7f76c41f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76c4144330 0x7f76c40a9e50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 --2- 172.21.15.82:0/1717784935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76c40714f0 0x7f76c40ab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 -- 172.21.15.82:0/1717784935 >> 172.21.15.82:0/1717784935 conn(0x7f76c40b1640 msgr2=0x7f76c415e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:26.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 -- 172.21.15.82:0/1717784935 shutdown_connections 2024-03-16T17:58:26.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.585+0000 7f76cb600640 1 -- 172.21.15.82:0/1717784935 wait complete. 2024-03-16T17:58:26.607 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:58:26.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.813+0000 7fbe4a800640 1 Processor -- start 2024-03-16T17:58:26.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.813+0000 7fbe4a800640 1 -- start start 2024-03-16T17:58:26.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.815+0000 7fbe4a800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe44159960 0x7fbe4406e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.815+0000 7fbe4a800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4415c370 con 0x7fbe440cca20 2024-03-16T17:58:26.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.815+0000 7fbe4a800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4415c650 con 0x7fbe4406e770 2024-03-16T17:58:26.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.815+0000 7fbe4a800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe44159d40 con 0x7fbe44159960 2024-03-16T17:58:26.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.815+0000 7fbe43400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe44159960 0x7fbe4406e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.815+0000 7fbe43400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe44159960 0x7fbe4406e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58678/0 (socket says 172.21.15.82:58678) 2024-03-16T17:58:26.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.815+0000 7fbe43400640 1 -- 172.21.15.82:0/3899668774 learned_addr learned my addr 172.21.15.82:0/3899668774 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:26.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.816+0000 7fbe43400640 1 -- 172.21.15.82:0/3899668774 >> v1:172.21.15.90:6789/0 conn(0x7fbe4406e770 legacy=0x7fbe440cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:26.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.816+0000 7fbe43400640 1 -- 172.21.15.82:0/3899668774 >> v1:172.21.15.82:6789/0 conn(0x7fbe440cca20 legacy=0x7fbe440cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.816+0000 7fbe43400640 1 -- 172.21.15.82:0/3899668774 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4415c930 con 0x7fbe44159960 2024-03-16T17:58:26.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.817+0000 7fbe43400640 1 --2- 172.21.15.82:0/3899668774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe44159960 0x7fbe4406e230 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009e50 tx=0x7fbe3402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f54894dd0c98d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:26.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.817+0000 7fbe42000640 1 -- 172.21.15.82:0/3899668774 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3402cb10 con 0x7fbe44159960 2024-03-16T17:58:26.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.817+0000 7fbe42000640 1 -- 172.21.15.82:0/3899668774 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3402cc70 con 0x7fbe44159960 2024-03-16T17:58:26.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.817+0000 7fbe42000640 1 -- 172.21.15.82:0/3899668774 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe340343f0 con 0x7fbe44159960 2024-03-16T17:58:26.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.818+0000 7fbe4a800640 1 -- 172.21.15.82:0/3899668774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe44159960 msgr2=0x7fbe4406e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.818+0000 7fbe4a800640 1 --2- 172.21.15.82:0/3899668774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe44159960 0x7fbe4406e230 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009e50 tx=0x7fbe3402bf70 comp rx=0 tx=0).stop 2024-03-16T17:58:26.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.818+0000 7fbe4a800640 1 -- 172.21.15.82:0/3899668774 shutdown_connections 2024-03-16T17:58:26.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.818+0000 7fbe4a800640 1 --2- 172.21.15.82:0/3899668774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe44159960 0x7fbe4406e230 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.818+0000 7fbe4a800640 1 -- 172.21.15.82:0/3899668774 >> 172.21.15.82:0/3899668774 conn(0x7fbe440b1640 msgr2=0x7fbe440b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:26.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.818+0000 7fbe4a800640 1 -- 172.21.15.82:0/3899668774 shutdown_connections 2024-03-16T17:58:26.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.818+0000 7fbe4a800640 1 -- 172.21.15.82:0/3899668774 wait complete. 2024-03-16T17:58:26.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.819+0000 7fbe4a800640 1 Processor -- start 2024-03-16T17:58:26.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.820+0000 7fbe4a800640 1 -- start start 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe4a800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe4406e770 0x7fbe441582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe4a800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbe440cca20 0x7fbe440ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe4a800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44159960 0x7fbe44134d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe4a800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4406ec20 con 0x7fbe4406e770 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe4a800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4415cff0 con 0x7fbe44159960 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe4a800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440cb090 con 0x7fbe440cca20 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe43400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe4406e770 0x7fbe441582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe43400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe4406e770 0x7fbe441582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58686/0 (socket says 172.21.15.82:58686) 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe43400640 1 -- 172.21.15.82:0/2406301966 learned_addr learned my addr 172.21.15.82:0/2406301966 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe43400640 1 -- 172.21.15.82:0/2406301966 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbe440cca20 msgr2=0x7fbe440ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe42a00640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbe440cca20 0x7fbe440ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe43400640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbe440cca20 0x7fbe440ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe43e00640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44159960 0x7fbe44134d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe43400640 1 -- 172.21.15.82:0/2406301966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44159960 msgr2=0x7fbe44134d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe43400640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44159960 0x7fbe44134d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe43400640 1 -- 172.21.15.82:0/2406301966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe34009b30 con 0x7fbe4406e770 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.821+0000 7fbe43e00640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44159960 0x7fbe44134d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.822+0000 7fbe43400640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe4406e770 0x7fbe441582f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbe34000f80 tx=0x7fbe34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.822+0000 7fbe3b400640 1 -- 172.21.15.82:0/2406301966 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3403e450 con 0x7fbe4406e770 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.822+0000 7fbe3b400640 1 -- 172.21.15.82:0/2406301966 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3403e5f0 con 0x7fbe4406e770 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.822+0000 7fbe3b400640 1 -- 172.21.15.82:0/2406301966 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3403e9a0 con 0x7fbe4406e770 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.822+0000 7fbe4a800640 1 -- 172.21.15.82:0/2406301966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe440b3730 con 0x7fbe4406e770 2024-03-16T17:58:26.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.822+0000 7fbe4a800640 1 -- 172.21.15.82:0/2406301966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe440b3cf0 con 0x7fbe4406e770 2024-03-16T17:58:26.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.823+0000 7fbe4a800640 1 -- 172.21.15.82:0/2406301966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe44067f30 con 0x7fbe4406e770 2024-03-16T17:58:26.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.824+0000 7fbe3b400640 1 -- 172.21.15.82:0/2406301966 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fbe3403eb00 con 0x7fbe4406e770 2024-03-16T17:58:26.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.825+0000 7fbe3b400640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbe2006b3b0 0x7fbe2006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:26.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.825+0000 7fbe3b400640 1 -- 172.21.15.82:0/2406301966 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbe340acc40 con 0x7fbe4406e770 2024-03-16T17:58:26.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.825+0000 7fbe42a00640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbe2006b3b0 0x7fbe2006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:26.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.826+0000 7fbe42a00640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbe2006b3b0 0x7fbe2006d870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbe440cd930 tx=0x7fbe2c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:26.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:26.830+0000 7fbe3b400640 1 -- 172.21.15.82:0/2406301966 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbe34077820 con 0x7fbe4406e770 2024-03-16T17:58:27.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.030+0000 7fbe4a800640 1 -- 172.21.15.82:0/2406301966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbe440c90d0 con 0x7fbe4406e770 2024-03-16T17:58:27.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.030+0000 7fbe3b400640 1 -- 172.21.15.82:0/2406301966 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbe3403edb0 con 0x7fbe4406e770 2024-03-16T17:58:27.034 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:58:27.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 -- 172.21.15.82:0/2406301966 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbe2006b3b0 msgr2=0x7fbe2006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbe2006b3b0 0x7fbe2006d870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbe440cd930 tx=0x7fbe2c009040 comp rx=0 tx=0).stop 2024-03-16T17:58:27.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 -- 172.21.15.82:0/2406301966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe4406e770 msgr2=0x7fbe441582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe4406e770 0x7fbe441582f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbe34000f80 tx=0x7fbe34004a40 comp rx=0 tx=0).stop 2024-03-16T17:58:27.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 -- 172.21.15.82:0/2406301966 shutdown_connections 2024-03-16T17:58:27.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbe2006b3b0 0x7fbe2006d870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44159960 0x7fbe44134d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbe440cca20 0x7fbe440ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 --2- 172.21.15.82:0/2406301966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe4406e770 0x7fbe441582f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 -- 172.21.15.82:0/2406301966 >> 172.21.15.82:0/2406301966 conn(0x7fbe440b1640 msgr2=0x7fbe4406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:27.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 -- 172.21.15.82:0/2406301966 shutdown_connections 2024-03-16T17:58:27.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.033+0000 7fbe4a800640 1 -- 172.21.15.82:0/2406301966 wait complete. 2024-03-16T17:58:27.055 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:58:27.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.256+0000 7f7859c00640 1 Processor -- start 2024-03-16T17:58:27.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.257+0000 7f7859c00640 1 -- start start 2024-03-16T17:58:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.258+0000 7f7859c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f78541454d0 0x7f7854160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.258+0000 7f7859c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406e650 0x7f78540cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.258+0000 7f7859c00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78541458b0 con 0x7f785406d900 2024-03-16T17:58:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.258+0000 7f7859c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78540cf3b0 con 0x7f78541454d0 2024-03-16T17:58:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.258+0000 7f7859c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78540cf690 con 0x7f785406e650 2024-03-16T17:58:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.258+0000 7f7853e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f78541454d0 0x7f7854160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.259+0000 7f7853e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f78541454d0 0x7f7854160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58692/0 (socket says 172.21.15.82:58692) 2024-03-16T17:58:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.259+0000 7f7853e00640 1 -- 172.21.15.82:0/158104970 learned_addr learned my addr 172.21.15.82:0/158104970 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.259+0000 7f784b400640 1 --2- 172.21.15.82:0/158104970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406e650 0x7f78540cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.259+0000 7f7853e00640 1 -- 172.21.15.82:0/158104970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406e650 msgr2=0x7f78540cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.259+0000 7f7853e00640 1 --2- 172.21.15.82:0/158104970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406e650 0x7f78540cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.259+0000 7f7853e00640 1 -- 172.21.15.82:0/158104970 >> v1:172.21.15.90:6789/0 conn(0x7f785406d900 legacy=0x7f78540b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.259+0000 7f7853e00640 1 -- 172.21.15.82:0/158104970 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785415f7a0 con 0x7f78541454d0 2024-03-16T17:58:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.259+0000 7f784b400640 1 --2- 172.21.15.82:0/158104970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406e650 0x7f78540cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.260+0000 7f7853e00640 1 --2- 172.21.15.82:0/158104970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78541454d0 0x7f7854160280 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7844009e50 tx=0x7f784402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841767f7a8fa0714 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.260+0000 7f7853400640 1 -- 172.21.15.82:0/158104970 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784402ca80 con 0x7f78541454d0 2024-03-16T17:58:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.260+0000 7f7853400640 1 -- 172.21.15.82:0/158104970 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f784402cbe0 con 0x7f78541454d0 2024-03-16T17:58:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.260+0000 7f7853400640 1 -- 172.21.15.82:0/158104970 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78440343f0 con 0x7f78541454d0 2024-03-16T17:58:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.261+0000 7f7859c00640 1 -- 172.21.15.82:0/158104970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78541454d0 msgr2=0x7f7854160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.261+0000 7f7859c00640 1 --2- 172.21.15.82:0/158104970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78541454d0 0x7f7854160280 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7844009e50 tx=0x7f784402bee0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.261+0000 7f7859c00640 1 -- 172.21.15.82:0/158104970 shutdown_connections 2024-03-16T17:58:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.261+0000 7f7859c00640 1 --2- 172.21.15.82:0/158104970 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406e650 0x7f78540cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.261+0000 7f7859c00640 1 --2- 172.21.15.82:0/158104970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78541454d0 0x7f7854160280 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.261+0000 7f7859c00640 1 -- 172.21.15.82:0/158104970 >> 172.21.15.82:0/158104970 conn(0x7f785406ae70 msgr2=0x7f785406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.261+0000 7f7859c00640 1 -- 172.21.15.82:0/158104970 shutdown_connections 2024-03-16T17:58:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.261+0000 7f7859c00640 1 -- 172.21.15.82:0/158104970 wait complete. 2024-03-16T17:58:27.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.263+0000 7f7859c00640 1 Processor -- start 2024-03-16T17:58:27.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.263+0000 7f7859c00640 1 -- start start 2024-03-16T17:58:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.264+0000 7f7859c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406d900 0x7f7854139e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.264+0000 7f7859c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785406e650 0x7f785415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.264+0000 7f7859c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78541454d0 0x7f785415ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.264+0000 7f7859c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7854141640 con 0x7f785406e650 2024-03-16T17:58:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.264+0000 7f7859c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785415df80 con 0x7f78541454d0 2024-03-16T17:58:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.264+0000 7f7859c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785415d370 con 0x7f785406d900 2024-03-16T17:58:27.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.264+0000 7f784b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785406e650 0x7f785415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.264+0000 7f784b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785406e650 0x7f785415a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58706/0 (socket says 172.21.15.82:58706) 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.264+0000 7f784b400640 1 -- 172.21.15.82:0/4061816006 learned_addr learned my addr 172.21.15.82:0/4061816006 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f7853e00640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406d900 0x7f7854139e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f7859200640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78541454d0 0x7f785415ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f784b400640 1 -- 172.21.15.82:0/4061816006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406d900 msgr2=0x7f7854139e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f784b400640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406d900 0x7f7854139e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f784b400640 1 -- 172.21.15.82:0/4061816006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78541454d0 msgr2=0x7f785415ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f784b400640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78541454d0 0x7f785415ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f784b400640 1 -- 172.21.15.82:0/4061816006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7844009b30 con 0x7f785406e650 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f7859200640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78541454d0 0x7f785415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f784b400640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785406e650 0x7f785415a830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c7f0 tx=0x7f783c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f7850c00640 1 -- 172.21.15.82:0/4061816006 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c007a30 con 0x7f785406e650 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f7850c00640 1 -- 172.21.15.82:0/4061816006 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783c007b90 con 0x7f785406e650 2024-03-16T17:58:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f7850c00640 1 -- 172.21.15.82:0/4061816006 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783c00ce40 con 0x7f785406e650 2024-03-16T17:58:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.265+0000 7f7859c00640 1 -- 172.21.15.82:0/4061816006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785415d650 con 0x7f785406e650 2024-03-16T17:58:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.266+0000 7f7859c00640 1 -- 172.21.15.82:0/4061816006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785415dc10 con 0x7f785406e650 2024-03-16T17:58:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.267+0000 7f7859c00640 1 -- 172.21.15.82:0/4061816006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785404f250 con 0x7f785406e650 2024-03-16T17:58:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.268+0000 7f7850c00640 1 -- 172.21.15.82:0/4061816006 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f783c0113f0 con 0x7f785406e650 2024-03-16T17:58:27.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.268+0000 7f7850c00640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f783006b3b0 0x7f783006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.269+0000 7f7850c00640 1 -- 172.21.15.82:0/4061816006 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f783c089e10 con 0x7f785406e650 2024-03-16T17:58:27.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.269+0000 7f7853e00640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f783006b3b0 0x7f783006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.269+0000 7f7853e00640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f783006b3b0 0x7f783006d870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7844035000 tx=0x7f7844030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:27.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.273+0000 7f7850c00640 1 -- 172.21.15.82:0/4061816006 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f783c08d030 con 0x7f785406e650 2024-03-16T17:58:27.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.474+0000 7f7859c00640 1 -- 172.21.15.82:0/4061816006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f785406e430 con 0x7f785406e650 2024-03-16T17:58:27.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.475+0000 7f7850c00640 1 -- 172.21.15.82:0/4061816006 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f783c01f0a0 con 0x7f785406e650 2024-03-16T17:58:27.479 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:58:27.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.477+0000 7f7859c00640 1 -- 172.21.15.82:0/4061816006 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f783006b3b0 msgr2=0x7f783006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.477+0000 7f7859c00640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f783006b3b0 0x7f783006d870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7844035000 tx=0x7f7844030000 comp rx=0 tx=0).stop 2024-03-16T17:58:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.477+0000 7f7859c00640 1 -- 172.21.15.82:0/4061816006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785406e650 msgr2=0x7f785415a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.477+0000 7f7859c00640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785406e650 0x7f785415a830 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f783c00c7f0 tx=0x7f783c00cbf0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.478+0000 7f7859c00640 1 -- 172.21.15.82:0/4061816006 shutdown_connections 2024-03-16T17:58:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.478+0000 7f7859c00640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f783006b3b0 0x7f783006d870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.478+0000 7f7859c00640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78541454d0 0x7f785415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.478+0000 7f7859c00640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785406e650 0x7f785415a830 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.478+0000 7f7859c00640 1 --2- 172.21.15.82:0/4061816006 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f785406d900 0x7f7854139e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.478+0000 7f7859c00640 1 -- 172.21.15.82:0/4061816006 >> 172.21.15.82:0/4061816006 conn(0x7f785406ae70 msgr2=0x7f78540ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.478+0000 7f7859c00640 1 -- 172.21.15.82:0/4061816006 shutdown_connections 2024-03-16T17:58:27.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.478+0000 7f7859c00640 1 -- 172.21.15.82:0/4061816006 wait complete. 2024-03-16T17:58:27.498 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T17:58:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.699+0000 7fa594e00640 1 Processor -- start 2024-03-16T17:58:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.699+0000 7fa594e00640 1 -- start start 2024-03-16T17:58:27.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.700+0000 7fa594e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa5900ce870 0x7fa5900cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.700+0000 7fa594e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59006fb30 con 0x7fa5900cf190 2024-03-16T17:58:27.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.700+0000 7fa594e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59006f570 con 0x7fa5900d1630 2024-03-16T17:58:27.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.700+0000 7fa594e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5900ce4a0 con 0x7fa5900ce870 2024-03-16T17:58:27.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.700+0000 7fa58d400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa5900cf190 0x7fa5900cfaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39914/0 (socket says 172.21.15.82:39914) 2024-03-16T17:58:27.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.700+0000 7fa58d400640 1 -- 172.21.15.82:0/729412478 learned_addr learned my addr 172.21.15.82:0/729412478 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.701+0000 7fa58de00640 1 --2- 172.21.15.82:0/729412478 >> v2:172.21.15.90:3300/0 conn(0x7fa5900ce870 0x7fa5900cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.701+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141906010 0 0) 0x7fa59006fb30 con 0x7fa5900cf190 2024-03-16T17:58:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.701+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa574003520 con 0x7fa5900cf190 2024-03-16T17:58:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.701+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 84374035 0 0) 0x7fa574003520 con 0x7fa5900cf190 2024-03-16T17:58:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.701+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 >> v2:172.21.15.90:3300/0 conn(0x7fa5900ce870 msgr2=0x7fa5900cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.701+0000 7fa58ca00640 1 --2- 172.21.15.82:0/729412478 >> v2:172.21.15.90:3300/0 conn(0x7fa5900ce870 0x7fa5900cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.701+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 >> v1:172.21.15.90:6789/0 conn(0x7fa5900d1630 legacy=0x7fa5900d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.701+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5900d0910 con 0x7fa5900cf190 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.701+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fa5800019f0 con 0x7fa5900cf190 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.701+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 >> v1:172.21.15.82:6789/0 conn(0x7fa5900cf190 legacy=0x7fa5900cfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58ca00640 1 --2- 172.21.15.82:0/729412478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5740039c0 0x7fa574003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58ca00640 1 --2- 172.21.15.82:0/729412478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa574004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58ca00640 1 --2- 172.21.15.82:0/729412478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa574005180 0x7fa57400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900ce4a0 con 0x7fa5740047c0 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57400c080 con 0x7fa5740039c0 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57400c240 con 0x7fa574005180 2024-03-16T17:58:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58d400640 1 --2- 172.21.15.82:0/729412478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa574004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58de00640 1 --2- 172.21.15.82:0/729412478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5740039c0 0x7fa574003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58d400640 1 -- 172.21.15.82:0/729412478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa574005180 msgr2=0x7fa57400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:27.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58d400640 1 --2- 172.21.15.82:0/729412478 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa574005180 0x7fa57400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58d400640 1 -- 172.21.15.82:0/729412478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5740039c0 msgr2=0x7fa574003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:27.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58d400640 1 --2- 172.21.15.82:0/729412478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa5740039c0 0x7fa574003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58d400640 1 -- 172.21.15.82:0/729412478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa574003520 con 0x7fa5740047c0 2024-03-16T17:58:27.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.702+0000 7fa58e800640 1 -- 172.21.15.82:0/729412478 reap_dead start 2024-03-16T17:58:27.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.703+0000 7fa58d400640 1 --2- 172.21.15.82:0/729412478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa574004c40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa590072a70 tx=0x7fa5740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284b44daa9fe634a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:27.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.704+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580034630 con 0x7fa5740047c0 2024-03-16T17:58:27.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.704+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58002f070 con 0x7fa5740047c0 2024-03-16T17:58:27.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.704+0000 7fa58ca00640 1 -- 172.21.15.82:0/729412478 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa580034a50 con 0x7fa5740047c0 2024-03-16T17:58:27.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.704+0000 7fa594e00640 1 -- 172.21.15.82:0/729412478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 msgr2=0x7fa574004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.704+0000 7fa594e00640 1 --2- 172.21.15.82:0/729412478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa574004c40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa590072a70 tx=0x7fa5740057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.704+0000 7fa594e00640 1 -- 172.21.15.82:0/729412478 shutdown_connections 2024-03-16T17:58:27.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.704+0000 7fa594e00640 1 --2- 172.21.15.82:0/729412478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa574004c40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.704+0000 7fa594e00640 1 -- 172.21.15.82:0/729412478 >> 172.21.15.82:0/729412478 conn(0x7fa5900b3780 msgr2=0x7fa5900b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:27.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.704+0000 7fa594e00640 1 -- 172.21.15.82:0/729412478 shutdown_connections 2024-03-16T17:58:27.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.704+0000 7fa594e00640 1 -- 172.21.15.82:0/729412478 wait complete. 2024-03-16T17:58:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.706+0000 7fa594e00640 1 Processor -- start 2024-03-16T17:58:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.706+0000 7fa594e00640 1 -- start start 2024-03-16T17:58:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.707+0000 7fa594e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa5901f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.707+0000 7fa594e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa59015ed10 0x7fa5901e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.707+0000 7fa594e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5901f3dd0 0x7fa5901e83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.707+0000 7fa594e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59009fa00 con 0x7fa5740047c0 2024-03-16T17:58:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.707+0000 7fa594e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900c5dc0 con 0x7fa59015ed10 2024-03-16T17:58:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.707+0000 7fa594e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900c60a0 con 0x7fa5901f3dd0 2024-03-16T17:58:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.707+0000 7fa58de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa5901f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa58de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa5901f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58728/0 (socket says 172.21.15.82:58728) 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa58de00640 1 -- 172.21.15.82:0/3173953546 learned_addr learned my addr 172.21.15.82:0/3173953546 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa58e800640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5901f3dd0 0x7fa5901e83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa58d400640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa59015ed10 0x7fa5901e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa58de00640 1 -- 172.21.15.82:0/3173953546 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5901f3dd0 msgr2=0x7fa5901e83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa58de00640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5901f3dd0 0x7fa5901e83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa58de00640 1 -- 172.21.15.82:0/3173953546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa59015ed10 msgr2=0x7fa5901e7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa58de00640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa59015ed10 0x7fa5901e7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa58de00640 1 -- 172.21.15.82:0/3173953546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5900c6380 con 0x7fa5740047c0 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa58de00640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa5901f3890 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa5780116d0 tx=0x7fa578011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa586000640 1 -- 172.21.15.82:0/3173953546 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57800f9c0 con 0x7fa5740047c0 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa586000640 1 -- 172.21.15.82:0/3173953546 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57800fb20 con 0x7fa5740047c0 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa586000640 1 -- 172.21.15.82:0/3173953546 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57800fe90 con 0x7fa5740047c0 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.708+0000 7fa594e00640 1 -- 172.21.15.82:0/3173953546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5900c6660 con 0x7fa5740047c0 2024-03-16T17:58:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.709+0000 7fa594e00640 1 -- 172.21.15.82:0/3173953546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5900c6e60 con 0x7fa5740047c0 2024-03-16T17:58:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.710+0000 7fa594e00640 1 -- 172.21.15.82:0/3173953546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59009fde0 con 0x7fa5740047c0 2024-03-16T17:58:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.711+0000 7fa586000640 1 -- 172.21.15.82:0/3173953546 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa578015460 con 0x7fa5740047c0 2024-03-16T17:58:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.711+0000 7fa586000640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa56c06b3b0 0x7fa56c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.712+0000 7fa586000640 1 -- 172.21.15.82:0/3173953546 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa57808a350 con 0x7fa5740047c0 2024-03-16T17:58:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.712+0000 7fa58d400640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa56c06b3b0 0x7fa56c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:27.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.712+0000 7fa58d400640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa56c06b3b0 0x7fa56c06d870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa580034000 tx=0x7fa58002f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:27.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.716+0000 7fa586000640 1 -- 172.21.15.82:0/3173953546 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa578054f30 con 0x7fa5740047c0 2024-03-16T17:58:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.912+0000 7fa594e00640 1 -- 172.21.15.82:0/3173953546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa59004f330 con 0x7fa5740047c0 2024-03-16T17:58:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.913+0000 7fa586000640 1 -- 172.21.15.82:0/3173953546 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa5780585f0 con 0x7fa5740047c0 2024-03-16T17:58:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-16T17:58:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 -- 172.21.15.82:0/3173953546 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa56c06b3b0 msgr2=0x7fa56c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa56c06b3b0 0x7fa56c06d870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa580034000 tx=0x7fa58002f000 comp rx=0 tx=0).stop 2024-03-16T17:58:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 -- 172.21.15.82:0/3173953546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 msgr2=0x7fa5901f3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa5901f3890 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa5780116d0 tx=0x7fa578011b90 comp rx=0 tx=0).stop 2024-03-16T17:58:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 -- 172.21.15.82:0/3173953546 shutdown_connections 2024-03-16T17:58:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa56c06b3b0 0x7fa56c06d870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa5901f3dd0 0x7fa5901e83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa59015ed10 0x7fa5901e7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 --2- 172.21.15.82:0/3173953546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa5740047c0 0x7fa5901f3890 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 -- 172.21.15.82:0/3173953546 >> 172.21.15.82:0/3173953546 conn(0x7fa5900b3780 msgr2=0x7fa59015f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.915+0000 7fa594e00640 1 -- 172.21.15.82:0/3173953546 shutdown_connections 2024-03-16T17:58:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:27.916+0000 7fa594e00640 1 -- 172.21.15.82:0/3173953546 wait complete. 2024-03-16T17:58:27.937 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-16T17:58:27.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:27.937 DEBUG:teuthology.orchestra.run.smithi090:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-16T17:58:28.037 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T17:58:28.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:28.038 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-16T17:58:28.096 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:58:28.096 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:28.150 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:58:28.151 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:28.202 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:58:28.202 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:28.299 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:28.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.504+0000 7f9e8ee00640 1 Processor -- start 2024-03-16T17:58:28.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.505+0000 7f9e8ee00640 1 -- start start 2024-03-16T17:58:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.506+0000 7f9e8ee00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e8806efd0 0x7f9e8806e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.506+0000 7f9e8ee00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9e8815e7c0 0x7f9e8815ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.506+0000 7f9e8ee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8806d0e0 con 0x7f9e88155130 2024-03-16T17:58:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.506+0000 7f9e8ee00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e880cf880 con 0x7f9e8815e7c0 2024-03-16T17:58:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.506+0000 7f9e8ee00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8806d360 con 0x7f9e8806efd0 2024-03-16T17:58:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.506+0000 7f9e87e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9e88155130 0x7f9e8806e8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39928/0 (socket says 172.21.15.82:39928) 2024-03-16T17:58:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.506+0000 7f9e87e00640 1 -- 172.21.15.82:0/1098333554 learned_addr learned my addr 172.21.15.82:0/1098333554 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e7f400640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e8806efd0 0x7f9e8806e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e8ce00640 1 --2- 172.21.15.82:0/1098333554 >> v2:172.21.15.90:3300/0 conn(0x7f9e8815e7c0 0x7f9e8815ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863737447 0 0) 0x7f9e8806d0e0 con 0x7f9e88155130 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e6c003520 con 0x7f9e88155130 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3244920495 0 0) 0x7f9e6c003520 con 0x7f9e88155130 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e8806efd0 msgr2=0x7f9e8806e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e8806efd0 0x7f9e8806e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 >> v2:172.21.15.90:3300/0 conn(0x7f9e8815e7c0 msgr2=0x7f9e8815ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 --2- 172.21.15.82:0/1098333554 >> v2:172.21.15.90:3300/0 conn(0x7f9e8815e7c0 0x7f9e8815ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8806d640 con 0x7f9e88155130 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f9e78001670 con 0x7f9e88155130 2024-03-16T17:58:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e8ce00640 1 --2- 172.21.15.82:0/1098333554 >> v2:172.21.15.90:3300/0 conn(0x7f9e8815e7c0 0x7f9e8815ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 >> v1:172.21.15.82:6789/0 conn(0x7f9e88155130 legacy=0x7f9e8806e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 0x7f9e6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.507+0000 7f9e87400640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e8806efd0 0x7f9e6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e87400640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c004d00 0x7f9e6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880cf880 con 0x7f9e6c003970 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8806d360 con 0x7f9e6c004d00 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c00ba30 con 0x7f9e8806efd0 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e87e00640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 0x7f9e6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e8ce00640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c004d00 0x7f9e6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e7f400640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e8806efd0 0x7f9e6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e87e00640 1 -- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e8806efd0 msgr2=0x7f9e6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e87e00640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e8806efd0 0x7f9e6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e87e00640 1 -- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c004d00 msgr2=0x7f9e6c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e87e00640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c004d00 0x7f9e6c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e87e00640 1 -- 172.21.15.82:0/1098333554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c003520 con 0x7f9e6c003970 2024-03-16T17:58:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e8ce00640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e6c004d00 0x7f9e6c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.508+0000 7f9e7f400640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e8806efd0 0x7f9e6c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.509+0000 7f9e87e00640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 0x7f9e6c003d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800c160 tx=0x7f9e6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7002e247de223e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.510+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78005ad0 con 0x7f9e6c003970 2024-03-16T17:58:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.510+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e78005c30 con 0x7f9e6c003970 2024-03-16T17:58:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.510+0000 7f9e87400640 1 -- 172.21.15.82:0/1098333554 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7802c810 con 0x7f9e6c003970 2024-03-16T17:58:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.510+0000 7f9e8ee00640 1 -- 172.21.15.82:0/1098333554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 msgr2=0x7f9e6c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.510+0000 7f9e8ee00640 1 --2- 172.21.15.82:0/1098333554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 0x7f9e6c003d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800c160 tx=0x7f9e6c0057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.510+0000 7f9e8ce00640 1 -- 172.21.15.82:0/1098333554 reap_dead start 2024-03-16T17:58:28.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.510+0000 7f9e8ee00640 1 -- 172.21.15.82:0/1098333554 shutdown_connections 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.510+0000 7f9e8ee00640 1 -- 172.21.15.82:0/1098333554 >> 172.21.15.82:0/1098333554 conn(0x7f9e880b17b0 msgr2=0x7f9e880b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.510+0000 7f9e8ee00640 1 -- 172.21.15.82:0/1098333554 shutdown_connections 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.511+0000 7f9e8ee00640 1 -- 172.21.15.82:0/1098333554 wait complete. 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.512+0000 7f9e8ee00640 1 Processor -- start 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.512+0000 7f9e8ee00640 1 -- start start 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.513+0000 7f9e8ee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 0x7f9e881f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.513+0000 7f9e8ee00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e8814e8e0 0x7f9e8814fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.513+0000 7f9e8ee00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e88151a10 0x7f9e881605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.513+0000 7f9e8ee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8814f050 con 0x7f9e6c003970 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.513+0000 7f9e8ee00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8814ed70 con 0x7f9e8814e8e0 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.513+0000 7f9e8ee00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e881f7660 con 0x7f9e88151a10 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.513+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 0x7f9e881f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.513+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 0x7f9e881f8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58744/0 (socket says 172.21.15.82:58744) 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.513+0000 7f9e87e00640 1 -- 172.21.15.82:0/268256663 learned_addr learned my addr 172.21.15.82:0/268256663 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:28.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.514+0000 7f9e7f400640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e8814e8e0 0x7f9e8814fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.514+0000 7f9e8ce00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e88151a10 0x7f9e881605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.514+0000 7f9e87e00640 1 -- 172.21.15.82:0/268256663 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e88151a10 msgr2=0x7f9e881605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.514+0000 7f9e87e00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e88151a10 0x7f9e881605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.514+0000 7f9e87e00640 1 -- 172.21.15.82:0/268256663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e8814e8e0 msgr2=0x7f9e8814fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.514+0000 7f9e87e00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e8814e8e0 0x7f9e8814fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.514+0000 7f9e87e00640 1 -- 172.21.15.82:0/268256663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e881f7940 con 0x7f9e6c003970 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.514+0000 7f9e7f400640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e8814e8e0 0x7f9e8814fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.514+0000 7f9e87e00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 0x7f9e881f8140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800c280 tx=0x7f9e78000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.514+0000 7f9e84c00640 1 -- 172.21.15.82:0/268256663 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78039070 con 0x7f9e6c003970 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.515+0000 7f9e84c00640 1 -- 172.21.15.82:0/268256663 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7802cc20 con 0x7f9e6c003970 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.515+0000 7f9e84c00640 1 -- 172.21.15.82:0/268256663 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78007070 con 0x7f9e6c003970 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.515+0000 7f9e8ee00640 1 -- 172.21.15.82:0/268256663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e881f7c20 con 0x7f9e6c003970 2024-03-16T17:58:28.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.515+0000 7f9e8ee00640 1 -- 172.21.15.82:0/268256663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8814a790 con 0x7f9e6c003970 2024-03-16T17:58:28.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.516+0000 7f9e8ee00640 1 -- 172.21.15.82:0/268256663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e88071ff0 con 0x7f9e6c003970 2024-03-16T17:58:28.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.517+0000 7f9e84c00640 1 -- 172.21.15.82:0/268256663 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f9e78032070 con 0x7f9e6c003970 2024-03-16T17:58:28.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.517+0000 7f9e84c00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e6406b3b0 0x7f9e6406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.518+0000 7f9e84c00640 1 -- 172.21.15.82:0/268256663 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9e7803c040 con 0x7f9e6c003970 2024-03-16T17:58:28.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.518+0000 7f9e7f400640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e6406b3b0 0x7f9e6406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.518+0000 7f9e7f400640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e6406b3b0 0x7f9e6406d870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e8814acc0 tx=0x7f9e70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:28.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.522+0000 7f9e84c00640 1 -- 172.21.15.82:0/268256663 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e780ae030 con 0x7f9e6c003970 2024-03-16T17:58:28.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.719+0000 7f9e8ee00640 1 -- 172.21.15.82:0/268256663 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e88067f30 con 0x7f9e6c003970 2024-03-16T17:58:28.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.722+0000 7f9e84c00640 1 -- 172.21.15.82:0/268256663 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16865 (secure 0 0 0) 0x7f9e78075b90 con 0x7f9e6c003970 2024-03-16T17:58:28.726 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:28.726 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":23,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:27.214849+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:28.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.724+0000 7f9e8ee00640 1 -- 172.21.15.82:0/268256663 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e6406b3b0 msgr2=0x7f9e6406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.724+0000 7f9e8ee00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e6406b3b0 0x7f9e6406d870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e8814acc0 tx=0x7f9e70008040 comp rx=0 tx=0).stop 2024-03-16T17:58:28.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.724+0000 7f9e8ee00640 1 -- 172.21.15.82:0/268256663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 msgr2=0x7f9e881f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.724+0000 7f9e8ee00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 0x7f9e881f8140 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800c280 tx=0x7f9e78000fe0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.724+0000 7f9e8ee00640 1 -- 172.21.15.82:0/268256663 shutdown_connections 2024-03-16T17:58:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.724+0000 7f9e8ee00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e6406b3b0 0x7f9e6406d870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.724+0000 7f9e8ee00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e88151a10 0x7f9e881605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.724+0000 7f9e8ee00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e8814e8e0 0x7f9e8814fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.725+0000 7f9e8ee00640 1 --2- 172.21.15.82:0/268256663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e6c003970 0x7f9e881f8140 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.725+0000 7f9e8ee00640 1 -- 172.21.15.82:0/268256663 >> 172.21.15.82:0/268256663 conn(0x7f9e880b17b0 msgr2=0x7f9e880afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.725+0000 7f9e8ee00640 1 -- 172.21.15.82:0/268256663 shutdown_connections 2024-03-16T17:58:28.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.725+0000 7f9e8ee00640 1 -- 172.21.15.82:0/268256663 wait complete. 2024-03-16T17:58:28.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:28.747 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:28.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.956+0000 7f53bc400640 1 Processor -- start 2024-03-16T17:58:28.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.956+0000 7f53bc400640 1 -- start start 2024-03-16T17:58:28.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.958+0000 7f53bc400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b415e760 0x7f53b415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.958+0000 7f53bc400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b40ce8f0 con 0x7f53b40cc380 2024-03-16T17:58:28.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.958+0000 7f53bc400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b415f2d0 con 0x7f53b4144330 2024-03-16T17:58:28.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.958+0000 7f53bc400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b415f5b0 con 0x7f53b415e760 2024-03-16T17:58:28.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.958+0000 7f53ba400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f53b40cc380 0x7f53b40cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39944/0 (socket says 172.21.15.82:39944) 2024-03-16T17:58:28.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.958+0000 7f53ba400640 1 -- 172.21.15.82:0/1412604295 learned_addr learned my addr 172.21.15.82:0/1412604295 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:28.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.958+0000 7f53b9000640 1 --2- 172.21.15.82:0/1412604295 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b415e760 0x7f53b415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.959+0000 7f53b3e00640 1 -- 172.21.15.82:0/1412604295 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509148471 0 0) 0x7f53b40ce8f0 con 0x7f53b40cc380 2024-03-16T17:58:28.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.959+0000 7f53b3e00640 1 -- 172.21.15.82:0/1412604295 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f539c003520 con 0x7f53b40cc380 2024-03-16T17:58:28.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.959+0000 7f53b3e00640 1 -- 172.21.15.82:0/1412604295 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2659581503 0 0) 0x7f539c003520 con 0x7f53b40cc380 2024-03-16T17:58:28.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.959+0000 7f53b9000640 1 -- 172.21.15.82:0/1412604295 >> v1:172.21.15.90:6789/0 conn(0x7f53b4144330 legacy=0x7f53b4070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.959+0000 7f53b9000640 1 -- 172.21.15.82:0/1412604295 >> v1:172.21.15.82:6789/0 conn(0x7f53b40cc380 legacy=0x7f53b40cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.959+0000 7f53b9000640 1 -- 172.21.15.82:0/1412604295 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b40cacb0 con 0x7f53b415e760 2024-03-16T17:58:28.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.961+0000 7f53b9000640 1 --2- 172.21.15.82:0/1412604295 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b415e760 0x7f53b415ebe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f53a00097d0 tx=0x7f53a002b660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=349b291a9ae32a6b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:28.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.961+0000 7f53b3e00640 1 -- 172.21.15.82:0/1412604295 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ac0019f0 con 0x7f53b415e760 2024-03-16T17:58:28.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.961+0000 7f53b3e00640 1 -- 172.21.15.82:0/1412604295 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a002e050 con 0x7f53b415e760 2024-03-16T17:58:28.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.961+0000 7f53b3e00640 1 -- 172.21.15.82:0/1412604295 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0030730 con 0x7f53b415e760 2024-03-16T17:58:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.962+0000 7f53bc400640 1 -- 172.21.15.82:0/1412604295 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b415e760 msgr2=0x7f53b415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.962+0000 7f53bc400640 1 --2- 172.21.15.82:0/1412604295 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b415e760 0x7f53b415ebe0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f53a00097d0 tx=0x7f53a002b660 comp rx=0 tx=0).stop 2024-03-16T17:58:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.962+0000 7f53bc400640 1 -- 172.21.15.82:0/1412604295 shutdown_connections 2024-03-16T17:58:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.962+0000 7f53bc400640 1 --2- 172.21.15.82:0/1412604295 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b415e760 0x7f53b415ebe0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.962+0000 7f53bc400640 1 -- 172.21.15.82:0/1412604295 >> 172.21.15.82:0/1412604295 conn(0x7f53b40b1640 msgr2=0x7f53b40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.962+0000 7f53bc400640 1 -- 172.21.15.82:0/1412604295 shutdown_connections 2024-03-16T17:58:28.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.962+0000 7f53bc400640 1 -- 172.21.15.82:0/1412604295 wait complete. 2024-03-16T17:58:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.964+0000 7f53bc400640 1 Processor -- start 2024-03-16T17:58:28.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.964+0000 7f53bc400640 1 -- start start 2024-03-16T17:58:28.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53bc400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40cc380 0x7f53b40b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53bc400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b4144330 0x7f53b41575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53bc400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53b415e760 0x7f53b41f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53bc400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40b5830 con 0x7f53b40cc380 2024-03-16T17:58:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53bc400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b41310c0 con 0x7f53b415e760 2024-03-16T17:58:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53bc400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b40b4eb0 con 0x7f53b4144330 2024-03-16T17:58:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53b9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40cc380 0x7f53b40b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53b9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40cc380 0x7f53b40b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58748/0 (socket says 172.21.15.82:58748) 2024-03-16T17:58:28.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53b9a00640 1 -- 172.21.15.82:0/1887827662 learned_addr learned my addr 172.21.15.82:0/1887827662 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:28.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53b9000640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b4144330 0x7f53b41575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.965+0000 7f53ba400640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53b415e760 0x7f53b41f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53b9a00640 1 -- 172.21.15.82:0/1887827662 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b4144330 msgr2=0x7f53b41575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53b9a00640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b4144330 0x7f53b41575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53b9a00640 1 -- 172.21.15.82:0/1887827662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53b415e760 msgr2=0x7f53b41f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53b9a00640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53b415e760 0x7f53b41f0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53b9a00640 1 -- 172.21.15.82:0/1887827662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a0009400 con 0x7f53b40cc380 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53b9000640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b4144330 0x7f53b41575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53b9a00640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40cc380 0x7f53b40b2730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53a800c7b0 tx=0x7f53a800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53b1600640 1 -- 172.21.15.82:0/1887827662 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8005270 con 0x7f53b40cc380 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53b1600640 1 -- 172.21.15.82:0/1887827662 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a80053f0 con 0x7f53b40cc380 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53bc400640 1 -- 172.21.15.82:0/1887827662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b40b2d80 con 0x7f53b40cc380 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53b1600640 1 -- 172.21.15.82:0/1887827662 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a800a880 con 0x7f53b40cc380 2024-03-16T17:58:28.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.966+0000 7f53bc400640 1 -- 172.21.15.82:0/1887827662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b40b5450 con 0x7f53b40cc380 2024-03-16T17:58:28.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.968+0000 7f53bc400640 1 -- 172.21.15.82:0/1887827662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b40b5970 con 0x7f53b40cc380 2024-03-16T17:58:28.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.968+0000 7f53b1600640 1 -- 172.21.15.82:0/1887827662 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f53a800a9e0 con 0x7f53b40cc380 2024-03-16T17:58:28.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.969+0000 7f53b1600640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f539406b3b0 0x7f539406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:28.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.969+0000 7f53b1600640 1 -- 172.21.15.82:0/1887827662 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f53a8089350 con 0x7f53b40cc380 2024-03-16T17:58:28.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.970+0000 7f53b9000640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f539406b3b0 0x7f539406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:28.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.970+0000 7f53b9000640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f539406b3b0 0x7f539406d870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f53a0034000 tx=0x7f53a0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:28.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:28.974+0000 7f53b1600640 1 -- 172.21.15.82:0/1887827662 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f53a8053fb0 con 0x7f53b40cc380 2024-03-16T17:58:29.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.170+0000 7f53bc400640 1 -- 172.21.15.82:0/1887827662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53b4067f30 con 0x7f53b40cc380 2024-03-16T17:58:29.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.171+0000 7f53b1600640 1 -- 172.21.15.82:0/1887827662 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53a8017020 con 0x7f53b40cc380 2024-03-16T17:58:29.175 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:29.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.173+0000 7f53bc400640 1 -- 172.21.15.82:0/1887827662 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f539406b3b0 msgr2=0x7f539406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.173+0000 7f53bc400640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f539406b3b0 0x7f539406d870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f53a0034000 tx=0x7f53a0008030 comp rx=0 tx=0).stop 2024-03-16T17:58:29.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.173+0000 7f53bc400640 1 -- 172.21.15.82:0/1887827662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40cc380 msgr2=0x7f53b40b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.173+0000 7f53bc400640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40cc380 0x7f53b40b2730 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53a800c7b0 tx=0x7f53a800cc70 comp rx=0 tx=0).stop 2024-03-16T17:58:29.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.174+0000 7f53bc400640 1 -- 172.21.15.82:0/1887827662 shutdown_connections 2024-03-16T17:58:29.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.174+0000 7f53bc400640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f539406b3b0 0x7f539406d870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.174+0000 7f53bc400640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53b415e760 0x7f53b41f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.174+0000 7f53bc400640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53b4144330 0x7f53b41575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.174+0000 7f53bc400640 1 --2- 172.21.15.82:0/1887827662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53b40cc380 0x7f53b40b2730 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.174+0000 7f53bc400640 1 -- 172.21.15.82:0/1887827662 >> 172.21.15.82:0/1887827662 conn(0x7f53b40b1640 msgr2=0x7f53b40cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:29.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.174+0000 7f53bc400640 1 -- 172.21.15.82:0/1887827662 shutdown_connections 2024-03-16T17:58:29.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.174+0000 7f53bc400640 1 -- 172.21.15.82:0/1887827662 wait complete. 2024-03-16T17:58:29.195 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:58:29.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.407+0000 7f83bca00640 1 Processor -- start 2024-03-16T17:58:29.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.408+0000 7f83bca00640 1 -- start start 2024-03-16T17:58:29.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.409+0000 7f83bca00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 0x7f83b806e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.409+0000 7f83bca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83b806e770 0x7f83b80cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.409+0000 7f83bca00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b80ce550 con 0x7f83b80cc850 2024-03-16T17:58:29.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.409+0000 7f83bca00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b8159d40 con 0x7f83b806e770 2024-03-16T17:58:29.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.409+0000 7f83bca00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b815cff0 con 0x7f83b8159960 2024-03-16T17:58:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.410+0000 7f83b5400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83b806e770 0x7f83b80cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.410+0000 7f83b5400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83b806e770 0x7f83b80cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58758/0 (socket says 172.21.15.82:58758) 2024-03-16T17:58:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.410+0000 7f83b5400640 1 -- 172.21.15.82:0/4112720289 learned_addr learned my addr 172.21.15.82:0/4112720289 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.410+0000 7f83b5e00640 1 --2- 172.21.15.82:0/4112720289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 0x7f83b806e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.410+0000 7f83b5400640 1 -- 172.21.15.82:0/4112720289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 msgr2=0x7f83b806e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.410+0000 7f83b5400640 1 --2- 172.21.15.82:0/4112720289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 0x7f83b806e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.410+0000 7f83b5400640 1 -- 172.21.15.82:0/4112720289 >> v1:172.21.15.90:6789/0 conn(0x7f83b80cc850 legacy=0x7f83b815f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.410+0000 7f83b5400640 1 -- 172.21.15.82:0/4112720289 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b815c370 con 0x7f83b806e770 2024-03-16T17:58:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.410+0000 7f83b5e00640 1 --2- 172.21.15.82:0/4112720289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 0x7f83b806e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.411+0000 7f83b5400640 1 --2- 172.21.15.82:0/4112720289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b806e770 0x7f83b80cc310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009870 tx=0x7f83a402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bf4279204dc7e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.411+0000 7f83b4a00640 1 -- 172.21.15.82:0/4112720289 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a40317c0 con 0x7f83b806e770 2024-03-16T17:58:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.412+0000 7f83b4a00640 1 -- 172.21.15.82:0/4112720289 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a4031920 con 0x7f83b806e770 2024-03-16T17:58:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.412+0000 7f83b4a00640 1 -- 172.21.15.82:0/4112720289 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4031cb0 con 0x7f83b806e770 2024-03-16T17:58:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.412+0000 7f83bca00640 1 -- 172.21.15.82:0/4112720289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b806e770 msgr2=0x7f83b80cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.412+0000 7f83bca00640 1 --2- 172.21.15.82:0/4112720289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b806e770 0x7f83b80cc310 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009870 tx=0x7f83a402b860 comp rx=0 tx=0).stop 2024-03-16T17:58:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.412+0000 7f83bca00640 1 -- 172.21.15.82:0/4112720289 shutdown_connections 2024-03-16T17:58:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.412+0000 7f83bca00640 1 --2- 172.21.15.82:0/4112720289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b806e770 0x7f83b80cc310 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.412+0000 7f83bca00640 1 --2- 172.21.15.82:0/4112720289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 0x7f83b806e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.412+0000 7f83bca00640 1 -- 172.21.15.82:0/4112720289 >> 172.21.15.82:0/4112720289 conn(0x7f83b80b1640 msgr2=0x7f83b80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.412+0000 7f83bca00640 1 -- 172.21.15.82:0/4112720289 shutdown_connections 2024-03-16T17:58:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.412+0000 7f83bca00640 1 -- 172.21.15.82:0/4112720289 wait complete. 2024-03-16T17:58:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.414+0000 7f83bca00640 1 Processor -- start 2024-03-16T17:58:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.414+0000 7f83bca00640 1 -- start start 2024-03-16T17:58:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.415+0000 7f83bca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83b806e770 0x7f83b80abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.415+0000 7f83bca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b80cc850 0x7f83b81f7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.415+0000 7f83bca00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 0x7f83b81433b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.415+0000 7f83bca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80ac190 con 0x7f83b80cc850 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.415+0000 7f83bca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b81f7cb0 con 0x7f83b806e770 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.415+0000 7f83bca00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80ca2f0 con 0x7f83b8159960 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.415+0000 7f83b5400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b80cc850 0x7f83b81f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.415+0000 7f83b5400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b80cc850 0x7f83b81f7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58762/0 (socket says 172.21.15.82:58762) 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.415+0000 7f83b5400640 1 -- 172.21.15.82:0/1355724 learned_addr learned my addr 172.21.15.82:0/1355724 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83b5e00640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83b806e770 0x7f83b80abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83b6800640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 0x7f83b81433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83b5400640 1 -- 172.21.15.82:0/1355724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 msgr2=0x7f83b81433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83b5400640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 0x7f83b81433b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83b5400640 1 -- 172.21.15.82:0/1355724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83b806e770 msgr2=0x7f83b80abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83b5400640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83b806e770 0x7f83b80abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83b5400640 1 -- 172.21.15.82:0/1355724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a40094d0 con 0x7f83b80cc850 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83b5400640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b80cc850 0x7f83b81f7770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009870 tx=0x7f83a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83ae000640 1 -- 172.21.15.82:0/1355724 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4032960 con 0x7f83b80cc850 2024-03-16T17:58:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83ae000640 1 -- 172.21.15.82:0/1355724 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a4032ac0 con 0x7f83b80cc850 2024-03-16T17:58:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83ae000640 1 -- 172.21.15.82:0/1355724 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4032e70 con 0x7f83b80cc850 2024-03-16T17:58:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83bca00640 1 -- 172.21.15.82:0/1355724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b80ca5d0 con 0x7f83b80cc850 2024-03-16T17:58:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.416+0000 7f83bca00640 1 -- 172.21.15.82:0/1355724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b80cab90 con 0x7f83b80cc850 2024-03-16T17:58:29.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.417+0000 7f83bca00640 1 -- 172.21.15.82:0/1355724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b8067f30 con 0x7f83b80cc850 2024-03-16T17:58:29.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.418+0000 7f83ae000640 1 -- 172.21.15.82:0/1355724 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f83a404e450 con 0x7f83b80cc850 2024-03-16T17:58:29.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.419+0000 7f83ae000640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f839806b3e0 0x7f839806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.419+0000 7f83ae000640 1 -- 172.21.15.82:0/1355724 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f83a40abff0 con 0x7f83b80cc850 2024-03-16T17:58:29.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.419+0000 7f83b5e00640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f839806b3e0 0x7f839806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.420+0000 7f83b5e00640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f839806b3e0 0x7f839806d8a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f83b80c9090 tx=0x7f83a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:29.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.423+0000 7f83ae000640 1 -- 172.21.15.82:0/1355724 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f83a4076c10 con 0x7f83b80cc850 2024-03-16T17:58:29.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.618+0000 7f83bca00640 1 -- 172.21.15.82:0/1355724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83b80008d0 con 0x7f83b80cc850 2024-03-16T17:58:29.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.619+0000 7f83ae000640 1 -- 172.21.15.82:0/1355724 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83a407a2d0 con 0x7f83b80cc850 2024-03-16T17:58:29.622 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.621+0000 7f83bca00640 1 -- 172.21.15.82:0/1355724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f839806b3e0 msgr2=0x7f839806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.621+0000 7f83bca00640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f839806b3e0 0x7f839806d8a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f83b80c9090 tx=0x7f83a8008040 comp rx=0 tx=0).stop 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.621+0000 7f83bca00640 1 -- 172.21.15.82:0/1355724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b80cc850 msgr2=0x7f83b81f7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.621+0000 7f83bca00640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b80cc850 0x7f83b81f7770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009870 tx=0x7f83a40046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.622+0000 7f83bca00640 1 -- 172.21.15.82:0/1355724 shutdown_connections 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.622+0000 7f83bca00640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f839806b3e0 0x7f839806d8a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.622+0000 7f83bca00640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83b8159960 0x7f83b81433b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.622+0000 7f83bca00640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83b80cc850 0x7f83b81f7770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.622+0000 7f83bca00640 1 --2- 172.21.15.82:0/1355724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83b806e770 0x7f83b80abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.622+0000 7f83bca00640 1 -- 172.21.15.82:0/1355724 >> 172.21.15.82:0/1355724 conn(0x7f83b80b1640 msgr2=0x7f83b8160180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.622+0000 7f83bca00640 1 -- 172.21.15.82:0/1355724 shutdown_connections 2024-03-16T17:58:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.622+0000 7f83bca00640 1 -- 172.21.15.82:0/1355724 wait complete. 2024-03-16T17:58:29.642 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:58:29.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.850+0000 7f8494000640 1 Processor -- start 2024-03-16T17:58:29.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.850+0000 7f8494000640 1 -- start start 2024-03-16T17:58:29.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.852+0000 7f8494000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c147060 0x7f848c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.852+0000 7f8494000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c146da0 con 0x7f848c06ea70 2024-03-16T17:58:29.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.852+0000 7f8494000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c06d490 con 0x7f848c1469c0 2024-03-16T17:58:29.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.852+0000 7f8494000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c06d770 con 0x7f848c147060 2024-03-16T17:58:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.852+0000 7f8492000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f848c06ea70 0x7f848c0cddc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39946/0 (socket says 172.21.15.82:39946) 2024-03-16T17:58:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.852+0000 7f8492000640 1 -- 172.21.15.82:0/2092618874 learned_addr learned my addr 172.21.15.82:0/2092618874 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.852+0000 7f8490c00640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c147060 0x7f848c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.852+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687770207 0 0) 0x7f848c146da0 con 0x7f848c06ea70 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.852+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8474003520 con 0x7f848c06ea70 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1411965157 0 0) 0x7f8474003520 con 0x7f848c06ea70 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c147060 msgr2=0x7f848c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c147060 0x7f848c06e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 >> v1:172.21.15.90:6789/0 conn(0x7f848c1469c0 legacy=0x7f848c162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c06da50 con 0x7f848c06ea70 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f84840019f0 con 0x7f848c06ea70 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 >> v1:172.21.15.82:6789/0 conn(0x7f848c06ea70 legacy=0x7f848c0cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84740039a0 0x7f8474003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c147060 0x7f84740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8474004d00 0x7f8474005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c06d770 con 0x7f84740039a0 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847400ba90 con 0x7f8474004d00 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847400bc50 con 0x7f848c147060 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.853+0000 7f8491600640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84740039a0 0x7f8474003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.854+0000 7f8492000640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8474004d00 0x7f8474005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.854+0000 7f8490c00640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c147060 0x7f84740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.854+0000 7f8491600640 1 -- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c147060 msgr2=0x7f84740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.854+0000 7f8491600640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c147060 0x7f84740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.854+0000 7f8491600640 1 -- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8474004d00 msgr2=0x7f8474005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.854+0000 7f8491600640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8474004d00 0x7f8474005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.854+0000 7f8491600640 1 -- 172.21.15.82:0/2092618874 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8474003520 con 0x7f84740039a0 2024-03-16T17:58:29.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.855+0000 7f8491600640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84740039a0 0x7f8474003dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8480001720 tx=0x7f84740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0065586bfe5ff server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:29.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.855+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800052d0 con 0x7f84740039a0 2024-03-16T17:58:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.855+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848002e050 con 0x7f84740039a0 2024-03-16T17:58:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.855+0000 7f848be00640 1 -- 172.21.15.82:0/2092618874 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800035f0 con 0x7f84740039a0 2024-03-16T17:58:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.856+0000 7f8494000640 1 -- 172.21.15.82:0/2092618874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84740039a0 msgr2=0x7f8474003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.856+0000 7f8494000640 1 --2- 172.21.15.82:0/2092618874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84740039a0 0x7f8474003dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8480001720 tx=0x7f84740057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.856+0000 7f8492000640 1 -- 172.21.15.82:0/2092618874 reap_dead start 2024-03-16T17:58:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.856+0000 7f8494000640 1 -- 172.21.15.82:0/2092618874 shutdown_connections 2024-03-16T17:58:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.856+0000 7f8494000640 1 -- 172.21.15.82:0/2092618874 >> 172.21.15.82:0/2092618874 conn(0x7f848c0b1640 msgr2=0x7f848c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.856+0000 7f8494000640 1 -- 172.21.15.82:0/2092618874 shutdown_connections 2024-03-16T17:58:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.856+0000 7f8494000640 1 -- 172.21.15.82:0/2092618874 wait complete. 2024-03-16T17:58:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.858+0000 7f8494000640 1 Processor -- start 2024-03-16T17:58:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.858+0000 7f8494000640 1 -- start start 2024-03-16T17:58:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8494000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84740039a0 0x7f848c141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8494000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c142070 0x7f848c1424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8494000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c1e4b60 0x7f848c1e5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8494000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0b4500 con 0x7f848c1e4b60 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8494000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0b47e0 con 0x7f84740039a0 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8494000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0b4ac0 con 0x7f848c142070 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8492000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c1e4b60 0x7f848c1e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8492000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c1e4b60 0x7f848c1e5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58776/0 (socket says 172.21.15.82:58776) 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8492000640 1 -- 172.21.15.82:0/544090053 learned_addr learned my addr 172.21.15.82:0/544090053 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8490c00640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c142070 0x7f848c1424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8491600640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84740039a0 0x7f848c141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8492000640 1 -- 172.21.15.82:0/544090053 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c142070 msgr2=0x7f848c1424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8492000640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c142070 0x7f848c1424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8492000640 1 -- 172.21.15.82:0/544090053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84740039a0 msgr2=0x7f848c141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.859+0000 7f8492000640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84740039a0 0x7f848c141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:29.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.860+0000 7f8492000640 1 -- 172.21.15.82:0/544090053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c0b4da0 con 0x7f848c1e4b60 2024-03-16T17:58:29.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.860+0000 7f8492000640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c1e4b60 0x7f848c1e5010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f848c0cda20 tx=0x7f848400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:29.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.860+0000 7f8489600640 1 -- 172.21.15.82:0/544090053 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400ec10 con 0x7f848c1e4b60 2024-03-16T17:58:29.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.860+0000 7f8489600640 1 -- 172.21.15.82:0/544090053 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848400ed70 con 0x7f848c1e4b60 2024-03-16T17:58:29.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.860+0000 7f8489600640 1 -- 172.21.15.82:0/544090053 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400eee0 con 0x7f848c1e4b60 2024-03-16T17:58:29.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.860+0000 7f8494000640 1 -- 172.21.15.82:0/544090053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c0b5080 con 0x7f848c1e4b60 2024-03-16T17:58:29.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.860+0000 7f8494000640 1 -- 172.21.15.82:0/544090053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c0b4190 con 0x7f848c1e4b60 2024-03-16T17:58:29.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.861+0000 7f8494000640 1 -- 172.21.15.82:0/544090053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c0b5270 con 0x7f848c1e4b60 2024-03-16T17:58:29.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.862+0000 7f8489600640 1 -- 172.21.15.82:0/544090053 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8484013660 con 0x7f848c1e4b60 2024-03-16T17:58:29.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.863+0000 7f8489600640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f846c06b3b0 0x7f846c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:29.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.863+0000 7f8489600640 1 -- 172.21.15.82:0/544090053 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f848408a240 con 0x7f848c1e4b60 2024-03-16T17:58:29.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.863+0000 7f8491600640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f846c06b3b0 0x7f846c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:29.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.864+0000 7f8491600640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f846c06b3b0 0x7f846c06d870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8480033000 tx=0x7f8480009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:29.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:29.867+0000 7f8489600640 1 -- 172.21.15.82:0/544090053 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8484054d70 con 0x7f848c1e4b60 2024-03-16T17:58:30.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.048+0000 7f8494000640 1 -- 172.21.15.82:0/544090053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f848c067f30 con 0x7f848c1e4b60 2024-03-16T17:58:30.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.049+0000 7f8489600640 1 -- 172.21.15.82:0/544090053 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8484058430 con 0x7f848c1e4b60 2024-03-16T17:58:30.052 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:58:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 -- 172.21.15.82:0/544090053 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f846c06b3b0 msgr2=0x7f846c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f846c06b3b0 0x7f846c06d870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8480033000 tx=0x7f8480009030 comp rx=0 tx=0).stop 2024-03-16T17:58:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 -- 172.21.15.82:0/544090053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c1e4b60 msgr2=0x7f848c1e5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c1e4b60 0x7f848c1e5010 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f848c0cda20 tx=0x7f848400e530 comp rx=0 tx=0).stop 2024-03-16T17:58:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 -- 172.21.15.82:0/544090053 shutdown_connections 2024-03-16T17:58:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f846c06b3b0 0x7f846c06d870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f848c1e4b60 0x7f848c1e5010 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f848c142070 0x7f848c1424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 --2- 172.21.15.82:0/544090053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84740039a0 0x7f848c141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 -- 172.21.15.82:0/544090053 >> 172.21.15.82:0/544090053 conn(0x7f848c0b1640 msgr2=0x7f848c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:30.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 -- 172.21.15.82:0/544090053 shutdown_connections 2024-03-16T17:58:30.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.051+0000 7f8494000640 1 -- 172.21.15.82:0/544090053 wait complete. 2024-03-16T17:58:30.073 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T17:58:30.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.264+0000 7fb92ac00640 1 Processor -- start 2024-03-16T17:58:30.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.264+0000 7fb92ac00640 1 -- start start 2024-03-16T17:58:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.266+0000 7fb92ac00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb9240d2a50 0x7fb9240d4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.266+0000 7fb92ac00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92415de90 0x7fb92415e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:30.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.266+0000 7fb92ac00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9240cf4a0 con 0x7fb9240d1f60 2024-03-16T17:58:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.266+0000 7fb92ac00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9240b0aa0 con 0x7fb9240d2a50 2024-03-16T17:58:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.266+0000 7fb92ac00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9240cf780 con 0x7fb92415de90 2024-03-16T17:58:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.266+0000 7fb923400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb9240d2a50 0x7fb9240d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.266+0000 7fb923400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb9240d2a50 0x7fb9240d4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58784/0 (socket says 172.21.15.82:58784) 2024-03-16T17:58:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.266+0000 7fb923400640 1 -- 172.21.15.82:0/4025803617 learned_addr learned my addr 172.21.15.82:0/4025803617 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.266+0000 7fb928c00640 1 --2- 172.21.15.82:0/4025803617 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92415de90 0x7fb92415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.266+0000 7fb923400640 1 -- 172.21.15.82:0/4025803617 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92415de90 msgr2=0x7fb92415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.267+0000 7fb923400640 1 --2- 172.21.15.82:0/4025803617 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92415de90 0x7fb92415e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.267+0000 7fb923400640 1 -- 172.21.15.82:0/4025803617 >> v1:172.21.15.90:6789/0 conn(0x7fb9240d1f60 legacy=0x7fb9240d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:30.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.267+0000 7fb923400640 1 -- 172.21.15.82:0/4025803617 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9240cfa60 con 0x7fb9240d2a50 2024-03-16T17:58:30.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.268+0000 7fb923400640 1 --2- 172.21.15.82:0/4025803617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 0x7fb9240d4910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009870 tx=0x7fb90c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2391e8ac723e2f26 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.268+0000 7fb922a00640 1 -- 172.21.15.82:0/4025803617 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c031640 con 0x7fb9240d2a50 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.268+0000 7fb922a00640 1 -- 172.21.15.82:0/4025803617 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c0317a0 con 0x7fb9240d2a50 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.268+0000 7fb922a00640 1 -- 172.21.15.82:0/4025803617 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c031b30 con 0x7fb9240d2a50 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.268+0000 7fb92ac00640 1 -- 172.21.15.82:0/4025803617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 msgr2=0x7fb9240d4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.268+0000 7fb92ac00640 1 --2- 172.21.15.82:0/4025803617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 0x7fb9240d4910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009870 tx=0x7fb90c02b8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.269+0000 7fb92ac00640 1 -- 172.21.15.82:0/4025803617 shutdown_connections 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.269+0000 7fb92ac00640 1 --2- 172.21.15.82:0/4025803617 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92415de90 0x7fb92415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.269+0000 7fb92ac00640 1 --2- 172.21.15.82:0/4025803617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 0x7fb9240d4910 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.269+0000 7fb92ac00640 1 -- 172.21.15.82:0/4025803617 >> 172.21.15.82:0/4025803617 conn(0x7fb9240d3df0 msgr2=0x7fb9240d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.269+0000 7fb92ac00640 1 -- 172.21.15.82:0/4025803617 shutdown_connections 2024-03-16T17:58:30.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.269+0000 7fb92ac00640 1 -- 172.21.15.82:0/4025803617 wait complete. 2024-03-16T17:58:30.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.271+0000 7fb92ac00640 1 Processor -- start 2024-03-16T17:58:30.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.271+0000 7fb92ac00640 1 -- start start 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.272+0000 7fb92ac00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9240d1f60 0x7fb9240c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.272+0000 7fb92ac00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 0x7fb92415fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.272+0000 7fb92ac00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92415de90 0x7fb92415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.272+0000 7fb92ac00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9240c8230 con 0x7fb9240d2a50 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.272+0000 7fb92ac00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9240c7920 con 0x7fb9240d1f60 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.272+0000 7fb92ac00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9240cbf00 con 0x7fb92415de90 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.272+0000 7fb923400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 0x7fb92415fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.272+0000 7fb923400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 0x7fb92415fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58794/0 (socket says 172.21.15.82:58794) 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.272+0000 7fb923400640 1 -- 172.21.15.82:0/4294541493 learned_addr learned my addr 172.21.15.82:0/4294541493 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb928c00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92415de90 0x7fb92415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:30.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb923e00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9240d1f60 0x7fb9240c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb923400640 1 -- 172.21.15.82:0/4294541493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92415de90 msgr2=0x7fb92415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb923400640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92415de90 0x7fb92415b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb923400640 1 -- 172.21.15.82:0/4294541493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9240d1f60 msgr2=0x7fb9240c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb923400640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9240d1f60 0x7fb9240c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb923400640 1 -- 172.21.15.82:0/4294541493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c0094d0 con 0x7fb9240d2a50 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb923400640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 0x7fb92415fe20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb90c02b930 tx=0x7fb90c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb91be00640 1 -- 172.21.15.82:0/4294541493 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c03ac60 con 0x7fb9240d2a50 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb91be00640 1 -- 172.21.15.82:0/4294541493 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c03ade0 con 0x7fb9240d2a50 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb91be00640 1 -- 172.21.15.82:0/4294541493 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c042630 con 0x7fb9240d2a50 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb92ac00640 1 -- 172.21.15.82:0/4294541493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9240c7b70 con 0x7fb9240d2a50 2024-03-16T17:58:30.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.273+0000 7fb92ac00640 1 -- 172.21.15.82:0/4294541493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9240c8b90 con 0x7fb9240d2a50 2024-03-16T17:58:30.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.274+0000 7fb92ac00640 1 -- 172.21.15.82:0/4294541493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9240c8370 con 0x7fb9240d2a50 2024-03-16T17:58:30.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.275+0000 7fb91be00640 1 -- 172.21.15.82:0/4294541493 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fb90c02f050 con 0x7fb9240d2a50 2024-03-16T17:58:30.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.276+0000 7fb91be00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb90006b3b0 0x7fb90006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.276+0000 7fb91be00640 1 -- 172.21.15.82:0/4294541493 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb90c0ab820 con 0x7fb9240d2a50 2024-03-16T17:58:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.276+0000 7fb923e00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb90006b3b0 0x7fb90006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:30.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.277+0000 7fb923e00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb90006b3b0 0x7fb90006d870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb914003d90 tx=0x7fb914009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:30.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.281+0000 7fb91be00640 1 -- 172.21.15.82:0/4294541493 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb9240c8370 con 0x7fb9240d2a50 2024-03-16T17:58:30.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.473+0000 7fb92ac00640 1 -- 172.21.15.82:0/4294541493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb9240c8370 con 0x7fb9240d2a50 2024-03-16T17:58:30.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.474+0000 7fb91be00640 1 -- 172.21.15.82:0/4294541493 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb9240c8370 con 0x7fb9240d2a50 2024-03-16T17:58:30.477 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-16T17:58:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 -- 172.21.15.82:0/4294541493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb90006b3b0 msgr2=0x7fb90006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb90006b3b0 0x7fb90006d870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb914003d90 tx=0x7fb914009040 comp rx=0 tx=0).stop 2024-03-16T17:58:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 -- 172.21.15.82:0/4294541493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 msgr2=0x7fb92415fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 0x7fb92415fe20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb90c02b930 tx=0x7fb90c0046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 -- 172.21.15.82:0/4294541493 shutdown_connections 2024-03-16T17:58:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb90006b3b0 0x7fb90006d870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb92415de90 0x7fb92415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9240d2a50 0x7fb92415fe20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 --2- 172.21.15.82:0/4294541493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9240d1f60 0x7fb9240c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 -- 172.21.15.82:0/4294541493 >> 172.21.15.82:0/4294541493 conn(0x7fb9240d3df0 msgr2=0x7fb9240d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 -- 172.21.15.82:0/4294541493 shutdown_connections 2024-03-16T17:58:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:30.476+0000 7fb92ac00640 1 -- 172.21.15.82:0/4294541493 wait complete. 2024-03-16T17:58:30.496 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-16T17:58:30.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:30.497 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-16T17:58:30.592 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T17:58:30.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:30.592 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-16T17:58:30.641 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:58:30.641 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:30.696 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:58:30.696 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:30.741 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T17:58:30.741 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:30.830 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:31.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.051+0000 7f2d2b200640 1 Processor -- start 2024-03-16T17:58:31.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.052+0000 7f2d2b200640 1 -- start start 2024-03-16T17:58:31.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.053+0000 7f2d2b200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2d2c1595b0 0x7f2d2c0ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.053+0000 7f2d2b200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2d2c159990 0x7f2d2c0aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.053+0000 7f2d2b200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c0b3370 con 0x7f2d2c06d030 2024-03-16T17:58:31.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.053+0000 7f2d2b200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c15d1c0 con 0x7f2d2c159990 2024-03-16T17:58:31.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.053+0000 7f2d2b200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c06dd20 con 0x7f2d2c1595b0 2024-03-16T17:58:31.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.053+0000 7f2d21400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2d2c159990 0x7f2d2c0aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.054+0000 7f2d21400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2d2c159990 0x7f2d2c0aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58808/0 (socket says 172.21.15.82:58808) 2024-03-16T17:58:31.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.054+0000 7f2d21400640 1 -- 172.21.15.82:0/3283400963 learned_addr learned my addr 172.21.15.82:0/3283400963 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:31.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.054+0000 7f2d29e00640 1 --2- 172.21.15.82:0/3283400963 >> v2:172.21.15.90:3300/0 conn(0x7f2d2c1595b0 0x7f2d2c0ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.054+0000 7f2d21400640 1 -- 172.21.15.82:0/3283400963 >> v2:172.21.15.90:3300/0 conn(0x7f2d2c1595b0 msgr2=0x7f2d2c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:31.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.054+0000 7f2d21400640 1 --2- 172.21.15.82:0/3283400963 >> v2:172.21.15.90:3300/0 conn(0x7f2d2c1595b0 0x7f2d2c0ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.054+0000 7f2d21400640 1 -- 172.21.15.82:0/3283400963 >> v1:172.21.15.82:6789/0 conn(0x7f2d2c06d030 legacy=0x7f2d2c06d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.054+0000 7f2d21400640 1 -- 172.21.15.82:0/3283400963 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c15f7a0 con 0x7f2d2c159990 2024-03-16T17:58:31.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.055+0000 7f2d21400640 1 --2- 172.21.15.82:0/3283400963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c159990 0x7f2d2c0aec70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c06e8b0 tx=0x7f2d1402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43dab65088bf673 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:31.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.056+0000 7f2d29400640 1 -- 172.21.15.82:0/3283400963 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14030860 con 0x7f2d2c159990 2024-03-16T17:58:31.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.056+0000 7f2d29400640 1 -- 172.21.15.82:0/3283400963 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d140309c0 con 0x7f2d2c159990 2024-03-16T17:58:31.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.056+0000 7f2d29400640 1 -- 172.21.15.82:0/3283400963 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14030d50 con 0x7f2d2c159990 2024-03-16T17:58:31.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.056+0000 7f2d2b200640 1 -- 172.21.15.82:0/3283400963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c159990 msgr2=0x7f2d2c0aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.056+0000 7f2d2b200640 1 --2- 172.21.15.82:0/3283400963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c159990 0x7f2d2c0aec70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c06e8b0 tx=0x7f2d1402b9a0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.056+0000 7f2d2b200640 1 -- 172.21.15.82:0/3283400963 shutdown_connections 2024-03-16T17:58:31.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.056+0000 7f2d2b200640 1 --2- 172.21.15.82:0/3283400963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c159990 0x7f2d2c0aec70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.056+0000 7f2d2b200640 1 --2- 172.21.15.82:0/3283400963 >> v2:172.21.15.90:3300/0 conn(0x7f2d2c1595b0 0x7f2d2c0ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.056+0000 7f2d2b200640 1 -- 172.21.15.82:0/3283400963 >> 172.21.15.82:0/3283400963 conn(0x7f2d2c06ae70 msgr2=0x7f2d2c06b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:31.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.057+0000 7f2d2b200640 1 -- 172.21.15.82:0/3283400963 shutdown_connections 2024-03-16T17:58:31.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.057+0000 7f2d2b200640 1 -- 172.21.15.82:0/3283400963 wait complete. 2024-03-16T17:58:31.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.058+0000 7f2d2b200640 1 Processor -- start 2024-03-16T17:58:31.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.058+0000 7f2d2b200640 1 -- start start 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.059+0000 7f2d2b200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2d2c06d030 0x7f2d2c14a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.059+0000 7f2d2b200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1595b0 0x7f2d2c15a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.059+0000 7f2d2b200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d2c159990 0x7f2d2c0b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.059+0000 7f2d2b200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0af990 con 0x7f2d2c1595b0 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.059+0000 7f2d2b200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0afc70 con 0x7f2d2c159990 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.059+0000 7f2d2b200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0aff50 con 0x7f2d2c06d030 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d21400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1595b0 0x7f2d2c15a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d21400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1595b0 0x7f2d2c15a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58820/0 (socket says 172.21.15.82:58820) 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d21400640 1 -- 172.21.15.82:0/1824695920 learned_addr learned my addr 172.21.15.82:0/1824695920 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d2a800640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d2c159990 0x7f2d2c0b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d21400640 1 -- 172.21.15.82:0/1824695920 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2d2c06d030 msgr2=0x7f2d2c14a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d21400640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2d2c06d030 0x7f2d2c14a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d21400640 1 -- 172.21.15.82:0/1824695920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d2c159990 msgr2=0x7f2d2c0b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d21400640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d2c159990 0x7f2d2c0b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d21400640 1 -- 172.21.15.82:0/1824695920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140094d0 con 0x7f2d2c1595b0 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d2a800640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d2c159990 0x7f2d2c0b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.060+0000 7f2d21400640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1595b0 0x7f2d2c15a7b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2d1402b9e0 tx=0x7f2d14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.061+0000 7f2d22a00640 1 -- 172.21.15.82:0/1824695920 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14039d60 con 0x7f2d2c1595b0 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.061+0000 7f2d22a00640 1 -- 172.21.15.82:0/1824695920 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d14043400 con 0x7f2d2c1595b0 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.061+0000 7f2d2b200640 1 -- 172.21.15.82:0/1824695920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c0b0230 con 0x7f2d2c1595b0 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.061+0000 7f2d22a00640 1 -- 172.21.15.82:0/1824695920 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d140437b0 con 0x7f2d2c1595b0 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.061+0000 7f2d2b200640 1 -- 172.21.15.82:0/1824695920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c14a640 con 0x7f2d2c1595b0 2024-03-16T17:58:31.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.062+0000 7f2d2b200640 1 -- 172.21.15.82:0/1824695920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c0b0750 con 0x7f2d2c1595b0 2024-03-16T17:58:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.063+0000 7f2d22a00640 1 -- 172.21.15.82:0/1824695920 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f2d14043910 con 0x7f2d2c1595b0 2024-03-16T17:58:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.064+0000 7f2d22a00640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2d0806b3b0 0x7f2d0806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.064+0000 7f2d22a00640 1 -- 172.21.15.82:0/1824695920 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2d1403e070 con 0x7f2d2c1595b0 2024-03-16T17:58:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.064+0000 7f2d29e00640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2d0806b3b0 0x7f2d0806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.065+0000 7f2d29e00640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2d0806b3b0 0x7f2d0806d870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0098f0 tx=0x7f2d1c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:31.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.069+0000 7f2d22a00640 1 -- 172.21.15.82:0/1824695920 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2d2c0b0750 con 0x7f2d2c1595b0 2024-03-16T17:58:31.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.269+0000 7f2d2b200640 1 -- 172.21.15.82:0/1824695920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d2c0b0750 con 0x7f2d2c1595b0 2024-03-16T17:58:31.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.271+0000 7f2d22a00640 1 -- 172.21.15.82:0/1824695920 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16865 (secure 0 0 0) 0x7f2d2c0b0750 con 0x7f2d2c1595b0 2024-03-16T17:58:31.276 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:31.276 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":23,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:27.214849+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:31.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.274+0000 7f2d2b200640 1 -- 172.21.15.82:0/1824695920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2d0806b3b0 msgr2=0x7f2d0806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.274+0000 7f2d2b200640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2d0806b3b0 0x7f2d0806d870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0098f0 tx=0x7f2d1c008040 comp rx=0 tx=0).stop 2024-03-16T17:58:31.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.274+0000 7f2d2b200640 1 -- 172.21.15.82:0/1824695920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1595b0 msgr2=0x7f2d2c15a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.274+0000 7f2d2b200640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1595b0 0x7f2d2c15a7b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2d1402b9e0 tx=0x7f2d14004730 comp rx=0 tx=0).stop 2024-03-16T17:58:31.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.275+0000 7f2d2b200640 1 -- 172.21.15.82:0/1824695920 shutdown_connections 2024-03-16T17:58:31.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.275+0000 7f2d2b200640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2d0806b3b0 0x7f2d0806d870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.275+0000 7f2d2b200640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d2c159990 0x7f2d2c0b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.275+0000 7f2d2b200640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1595b0 0x7f2d2c15a7b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.275+0000 7f2d2b200640 1 --2- 172.21.15.82:0/1824695920 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2d2c06d030 0x7f2d2c14a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.275+0000 7f2d2b200640 1 -- 172.21.15.82:0/1824695920 >> 172.21.15.82:0/1824695920 conn(0x7f2d2c06ae70 msgr2=0x7f2d2c153d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:31.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.275+0000 7f2d2b200640 1 -- 172.21.15.82:0/1824695920 shutdown_connections 2024-03-16T17:58:31.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.275+0000 7f2d2b200640 1 -- 172.21.15.82:0/1824695920 wait complete. 2024-03-16T17:58:31.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:31.299 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:31.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.511+0000 7f6bff000640 1 Processor -- start 2024-03-16T17:58:31.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.512+0000 7f6bff000640 1 -- start start 2024-03-16T17:58:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.513+0000 7f6bff000640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6bf81454d0 0x7f6bf8160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.513+0000 7f6bff000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf806e650 0x7f6bf80cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.513+0000 7f6bff000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bf81458b0 con 0x7f6bf806d900 2024-03-16T17:58:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.513+0000 7f6bff000640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bf80cf3b0 con 0x7f6bf81454d0 2024-03-16T17:58:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.513+0000 7f6bff000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf80cf690 con 0x7f6bf806e650 2024-03-16T17:58:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.514+0000 7f6bfdc00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6bf81454d0 0x7f6bf8160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.514+0000 7f6bfd200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf806e650 0x7f6bf80cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.514+0000 7f6bfdc00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f6bf81454d0 0x7f6bf8160280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:49038/0 (socket says 172.21.15.82:49038) 2024-03-16T17:58:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.514+0000 7f6bfdc00640 1 -- 172.21.15.82:0/4272301913 learned_addr learned my addr 172.21.15.82:0/4272301913 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.514+0000 7f6bfe600640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f6bf806d900 0x7f6bf80b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.82:48950/0 (socket says 172.21.15.82:48950) 2024-03-16T17:58:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.515+0000 7f6bfd200640 1 -- 172.21.15.82:0/4272301913 >> v2:172.21.15.90:3300/0 conn(0x7f6bf81454d0 msgr2=0x7f6bf8160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.515+0000 7f6bfd200640 1 --2- 172.21.15.82:0/4272301913 >> v2:172.21.15.90:3300/0 conn(0x7f6bf81454d0 0x7f6bf8160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.515+0000 7f6bfd200640 1 -- 172.21.15.82:0/4272301913 >> v1:172.21.15.90:6789/0 conn(0x7f6bf806d900 legacy=0x7f6bf80b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.515+0000 7f6bfd200640 1 -- 172.21.15.82:0/4272301913 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf815f7a0 con 0x7f6bf806e650 2024-03-16T17:58:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.516+0000 7f6bfd200640 1 --2- 172.21.15.82:0/4272301913 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf806e650 0x7f6bf80cd6f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6be40097a0 tx=0x7f6be402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7211d38e186a8e4c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:31.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.517+0000 7f6bf7e00640 1 -- 172.21.15.82:0/4272301913 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be402d040 con 0x7f6bf806e650 2024-03-16T17:58:31.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.517+0000 7f6bf7e00640 1 -- 172.21.15.82:0/4272301913 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be4031810 con 0x7f6bf806e650 2024-03-16T17:58:31.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.517+0000 7f6bf7e00640 1 -- 172.21.15.82:0/4272301913 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4031ba0 con 0x7f6bf806e650 2024-03-16T17:58:31.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.517+0000 7f6bff000640 1 -- 172.21.15.82:0/4272301913 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf806e650 msgr2=0x7f6bf80cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.517+0000 7f6bff000640 1 --2- 172.21.15.82:0/4272301913 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf806e650 0x7f6bf80cd6f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6be40097a0 tx=0x7f6be402b8b0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.518+0000 7f6bff000640 1 -- 172.21.15.82:0/4272301913 shutdown_connections 2024-03-16T17:58:31.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.518+0000 7f6bff000640 1 --2- 172.21.15.82:0/4272301913 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf806e650 0x7f6bf80cd6f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.518+0000 7f6bff000640 1 --2- 172.21.15.82:0/4272301913 >> v2:172.21.15.90:3300/0 conn(0x7f6bf81454d0 0x7f6bf8160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.518+0000 7f6bff000640 1 -- 172.21.15.82:0/4272301913 >> 172.21.15.82:0/4272301913 conn(0x7f6bf806ae70 msgr2=0x7f6bf806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:31.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.518+0000 7f6bff000640 1 -- 172.21.15.82:0/4272301913 shutdown_connections 2024-03-16T17:58:31.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.518+0000 7f6bff000640 1 -- 172.21.15.82:0/4272301913 wait complete. 2024-03-16T17:58:31.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.520+0000 7f6bff000640 1 Processor -- start 2024-03-16T17:58:31.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.520+0000 7f6bff000640 1 -- start start 2024-03-16T17:58:31.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.521+0000 7f6bff000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf806d900 0x7f6bf8142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.521+0000 7f6bff000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf806e650 0x7f6bf8131d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.521+0000 7f6bff000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf81454d0 0x7f6bf80722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.521+0000 7f6bff000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf81322b0 con 0x7f6bf806d900 2024-03-16T17:58:31.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.521+0000 7f6bff000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf80727f0 con 0x7f6bf806e650 2024-03-16T17:58:31.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.521+0000 7f6bff000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf81430a0 con 0x7f6bf81454d0 2024-03-16T17:58:31.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfdc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf806d900 0x7f6bf8142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfdc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf806d900 0x7f6bf8142ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58826/0 (socket says 172.21.15.82:58826) 2024-03-16T17:58:31.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfdc00640 1 -- 172.21.15.82:0/949143195 learned_addr learned my addr 172.21.15.82:0/949143195 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfe600640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf81454d0 0x7f6bf80722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfd200640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf806e650 0x7f6bf8131d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfdc00640 1 -- 172.21.15.82:0/949143195 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf81454d0 msgr2=0x7f6bf80722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfdc00640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf81454d0 0x7f6bf80722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfdc00640 1 -- 172.21.15.82:0/949143195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf806e650 msgr2=0x7f6bf8131d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfdc00640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf806e650 0x7f6bf8131d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfdc00640 1 -- 172.21.15.82:0/949143195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be4009400 con 0x7f6bf806d900 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfd200640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf806e650 0x7f6bf8131d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.522+0000 7f6bfdc00640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf806d900 0x7f6bf8142ac0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009fd0 tx=0x7f6bec00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.523+0000 7f6bf5600640 1 -- 172.21.15.82:0/949143195 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec00c940 con 0x7f6bf806d900 2024-03-16T17:58:31.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.523+0000 7f6bf5600640 1 -- 172.21.15.82:0/949143195 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bec00caa0 con 0x7f6bf806d900 2024-03-16T17:58:31.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.523+0000 7f6bf5600640 1 -- 172.21.15.82:0/949143195 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bec00ce30 con 0x7f6bf806d900 2024-03-16T17:58:31.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.523+0000 7f6bff000640 1 -- 172.21.15.82:0/949143195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf8143380 con 0x7f6bf806d900 2024-03-16T17:58:31.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.523+0000 7f6bff000640 1 -- 172.21.15.82:0/949143195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf8143940 con 0x7f6bf806d900 2024-03-16T17:58:31.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.524+0000 7f6bff000640 1 -- 172.21.15.82:0/949143195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf804f250 con 0x7f6bf806d900 2024-03-16T17:58:31.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.525+0000 7f6bf5600640 1 -- 172.21.15.82:0/949143195 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f6bec013470 con 0x7f6bf806d900 2024-03-16T17:58:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.526+0000 7f6bf5600640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6bd806b3b0 0x7f6bd806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.526+0000 7f6bf5600640 1 -- 172.21.15.82:0/949143195 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6bec089da0 con 0x7f6bf806d900 2024-03-16T17:58:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.526+0000 7f6bfd200640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6bd806b3b0 0x7f6bd806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.527+0000 7f6bfd200640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6bd806b3b0 0x7f6bd806d870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6be4036000 tx=0x7f6be403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:31.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.530+0000 7f6bf5600640 1 -- 172.21.15.82:0/949143195 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6bec054a00 con 0x7f6bf806d900 2024-03-16T17:58:31.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.727+0000 7f6bff000640 1 -- 172.21.15.82:0/949143195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6bf80b4c10 con 0x7f6bf806d900 2024-03-16T17:58:31.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.728+0000 7f6bf5600640 1 -- 172.21.15.82:0/949143195 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6bec0170a0 con 0x7f6bf806d900 2024-03-16T17:58:31.732 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 -- 172.21.15.82:0/949143195 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6bd806b3b0 msgr2=0x7f6bd806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6bd806b3b0 0x7f6bd806d870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6be4036000 tx=0x7f6be403c040 comp rx=0 tx=0).stop 2024-03-16T17:58:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 -- 172.21.15.82:0/949143195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf806d900 msgr2=0x7f6bf8142ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf806d900 0x7f6bf8142ac0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009fd0 tx=0x7f6bec00ed40 comp rx=0 tx=0).stop 2024-03-16T17:58:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 -- 172.21.15.82:0/949143195 shutdown_connections 2024-03-16T17:58:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f6bd806b3b0 0x7f6bd806d870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f6bf81454d0 0x7f6bf80722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf806e650 0x7f6bf8131d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 --2- 172.21.15.82:0/949143195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6bf806d900 0x7f6bf8142ac0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 -- 172.21.15.82:0/949143195 >> 172.21.15.82:0/949143195 conn(0x7f6bf806ae70 msgr2=0x7f6bf80cc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:31.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 -- 172.21.15.82:0/949143195 shutdown_connections 2024-03-16T17:58:31.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.731+0000 7f6bff000640 1 -- 172.21.15.82:0/949143195 wait complete. 2024-03-16T17:58:31.755 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:58:31.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.954+0000 7f585c800640 1 Processor -- start 2024-03-16T17:58:31.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.955+0000 7f585c800640 1 -- start start 2024-03-16T17:58:31.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.956+0000 7f585c800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f585415e040 0x7f585415e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.956+0000 7f585c800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58540d48e0 0x7f58540d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.956+0000 7f585c800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58540b33c0 con 0x7f58540d28e0 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.956+0000 7f585c800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58540d2d10 con 0x7f58540d48e0 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.956+0000 7f585c800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58540d1cd0 con 0x7f585415e040 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.956+0000 7f5859400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58540d48e0 0x7f58540d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.957+0000 7f5859400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58540d48e0 0x7f58540d23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58832/0 (socket says 172.21.15.82:58832) 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.957+0000 7f5859400640 1 -- 172.21.15.82:0/3619731119 learned_addr learned my addr 172.21.15.82:0/3619731119 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.957+0000 7f585a800640 1 --1- 172.21.15.82:0/3619731119 >> v1:172.21.15.82:6789/0 conn(0x7f58540d28e0 0x7f58540b3e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39964/0 (socket says 172.21.15.82:39964) 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.957+0000 7f5859e00640 1 --2- 172.21.15.82:0/3619731119 >> v2:172.21.15.90:3300/0 conn(0x7f585415e040 0x7f585415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.957+0000 7f5859400640 1 -- 172.21.15.82:0/3619731119 >> v2:172.21.15.90:3300/0 conn(0x7f585415e040 msgr2=0x7f585415e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.957+0000 7f5859400640 1 --2- 172.21.15.82:0/3619731119 >> v2:172.21.15.90:3300/0 conn(0x7f585415e040 0x7f585415e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.957+0000 7f5859400640 1 -- 172.21.15.82:0/3619731119 >> v1:172.21.15.82:6789/0 conn(0x7f58540d28e0 legacy=0x7f58540b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.957+0000 7f5859400640 1 -- 172.21.15.82:0/3619731119 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58540d1fb0 con 0x7f58540d48e0 2024-03-16T17:58:31.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.958+0000 7f5859400640 1 --2- 172.21.15.82:0/3619731119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d48e0 0x7f58540d23a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f58540b69b0 tx=0x7f584402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11d63681c60ad1c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:31.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.958+0000 7f5858a00640 1 -- 172.21.15.82:0/3619731119 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844031870 con 0x7f58540d48e0 2024-03-16T17:58:31.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.959+0000 7f5858a00640 1 -- 172.21.15.82:0/3619731119 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58440319d0 con 0x7f58540d48e0 2024-03-16T17:58:31.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.959+0000 7f5858a00640 1 -- 172.21.15.82:0/3619731119 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844031d60 con 0x7f58540d48e0 2024-03-16T17:58:31.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.959+0000 7f585c800640 1 -- 172.21.15.82:0/3619731119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d48e0 msgr2=0x7f58540d23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.959+0000 7f585c800640 1 --2- 172.21.15.82:0/3619731119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d48e0 0x7f58540d23a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f58540b69b0 tx=0x7f584402b910 comp rx=0 tx=0).stop 2024-03-16T17:58:31.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.959+0000 7f585c800640 1 -- 172.21.15.82:0/3619731119 shutdown_connections 2024-03-16T17:58:31.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.959+0000 7f585c800640 1 --2- 172.21.15.82:0/3619731119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d48e0 0x7f58540d23a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.959+0000 7f585c800640 1 --2- 172.21.15.82:0/3619731119 >> v2:172.21.15.90:3300/0 conn(0x7f585415e040 0x7f585415e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.959+0000 7f585c800640 1 -- 172.21.15.82:0/3619731119 >> 172.21.15.82:0/3619731119 conn(0x7f58540d3dc0 msgr2=0x7f58540d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:31.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.959+0000 7f585c800640 1 -- 172.21.15.82:0/3619731119 shutdown_connections 2024-03-16T17:58:31.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.959+0000 7f585c800640 1 -- 172.21.15.82:0/3619731119 wait complete. 2024-03-16T17:58:31.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.961+0000 7f585c800640 1 Processor -- start 2024-03-16T17:58:31.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.961+0000 7f585c800640 1 -- start start 2024-03-16T17:58:31.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.962+0000 7f585c800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d28e0 0x7f58540b69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.962+0000 7f585c800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585415e040 0x7f58540b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.962+0000 7f585c800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58540b4290 0x7f58540b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.962+0000 7f585c800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58540b4c80 con 0x7f58540d28e0 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.962+0000 7f585c800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5854152af0 con 0x7f585415e040 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.962+0000 7f585c800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58541f81a0 con 0x7f58540b4290 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.962+0000 7f5859e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d28e0 0x7f58540b69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.962+0000 7f5859e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d28e0 0x7f58540b69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58844/0 (socket says 172.21.15.82:58844) 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.962+0000 7f5859e00640 1 -- 172.21.15.82:0/2500889014 learned_addr learned my addr 172.21.15.82:0/2500889014 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f5859400640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585415e040 0x7f58540b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f585a800640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58540b4290 0x7f58540b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f5859e00640 1 -- 172.21.15.82:0/2500889014 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58540b4290 msgr2=0x7f58540b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f5859e00640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58540b4290 0x7f58540b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f5859e00640 1 -- 172.21.15.82:0/2500889014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585415e040 msgr2=0x7f58540b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f5859e00640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585415e040 0x7f58540b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:31.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f5859e00640 1 -- 172.21.15.82:0/2500889014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440094d0 con 0x7f58540d28e0 2024-03-16T17:58:31.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f5859400640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585415e040 0x7f58540b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:31.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f5859e00640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d28e0 0x7f58540b69b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f584800d650 tx=0x7f584800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:31.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f584e000640 1 -- 172.21.15.82:0/2500889014 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584800b660 con 0x7f58540d28e0 2024-03-16T17:58:31.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f584e000640 1 -- 172.21.15.82:0/2500889014 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584800b7c0 con 0x7f58540d28e0 2024-03-16T17:58:31.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f584e000640 1 -- 172.21.15.82:0/2500889014 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584800bb70 con 0x7f58540d28e0 2024-03-16T17:58:31.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f585c800640 1 -- 172.21.15.82:0/2500889014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58541f7240 con 0x7f58540d28e0 2024-03-16T17:58:31.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.963+0000 7f585c800640 1 -- 172.21.15.82:0/2500889014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58541f7800 con 0x7f58540d28e0 2024-03-16T17:58:31.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.965+0000 7f585c800640 1 -- 172.21.15.82:0/2500889014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5854067f30 con 0x7f58540d28e0 2024-03-16T17:58:31.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.966+0000 7f584e000640 1 -- 172.21.15.82:0/2500889014 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f584800bcd0 con 0x7f58540d28e0 2024-03-16T17:58:31.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.966+0000 7f584e000640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f583806b3b0 0x7f583806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:31.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.966+0000 7f584e000640 1 -- 172.21.15.82:0/2500889014 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f584808a120 con 0x7f58540d28e0 2024-03-16T17:58:31.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.967+0000 7f5859400640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f583806b3b0 0x7f583806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:31.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.967+0000 7f5859400640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f583806b3b0 0x7f583806d870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5844036000 tx=0x7f584403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:31.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:31.971+0000 7f584e000640 1 -- 172.21.15.82:0/2500889014 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5848054ce0 con 0x7f58540d28e0 2024-03-16T17:58:32.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.151+0000 7f585c800640 1 -- 172.21.15.82:0/2500889014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58540b8910 con 0x7f58540d28e0 2024-03-16T17:58:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.152+0000 7f584e000640 1 -- 172.21.15.82:0/2500889014 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58480583a0 con 0x7f58540d28e0 2024-03-16T17:58:32.156 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:58:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 -- 172.21.15.82:0/2500889014 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f583806b3b0 msgr2=0x7f583806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f583806b3b0 0x7f583806d870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5844036000 tx=0x7f584403c040 comp rx=0 tx=0).stop 2024-03-16T17:58:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 -- 172.21.15.82:0/2500889014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d28e0 msgr2=0x7f58540b69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d28e0 0x7f58540b69b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f584800d650 tx=0x7f584800db10 comp rx=0 tx=0).stop 2024-03-16T17:58:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 -- 172.21.15.82:0/2500889014 shutdown_connections 2024-03-16T17:58:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f583806b3b0 0x7f583806d870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58540b4290 0x7f58540b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f585415e040 0x7f58540b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 --2- 172.21.15.82:0/2500889014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58540d28e0 0x7f58540b69b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 -- 172.21.15.82:0/2500889014 >> 172.21.15.82:0/2500889014 conn(0x7f58540d3dc0 msgr2=0x7f58540ccea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 -- 172.21.15.82:0/2500889014 shutdown_connections 2024-03-16T17:58:32.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.155+0000 7f585c800640 1 -- 172.21.15.82:0/2500889014 wait complete. 2024-03-16T17:58:32.177 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:58:32.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.375+0000 7ffa17e00640 1 Processor -- start 2024-03-16T17:58:32.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.375+0000 7ffa17e00640 1 -- start start 2024-03-16T17:58:32.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.376+0000 7ffa17e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa1806e010 0x7ffa18160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.376+0000 7ffa17e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa181458b0 con 0x7ffa1806e650 2024-03-16T17:58:32.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.377+0000 7ffa17e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa180cf3b0 con 0x7ffa181454d0 2024-03-16T17:58:32.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.377+0000 7ffa17e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa180cf690 con 0x7ffa1806e010 2024-03-16T17:58:32.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.377+0000 7ffa16000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa1806e010 0x7ffa18160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.377+0000 7ffa17400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffa1806e650 0x7ffa180b29b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39970/0 (socket says 172.21.15.82:39970) 2024-03-16T17:58:32.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.377+0000 7ffa17400640 1 -- 172.21.15.82:0/3724273814 learned_addr learned my addr 172.21.15.82:0/3724273814 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:32.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.377+0000 7ffa16000640 1 -- 172.21.15.82:0/3724273814 >> v1:172.21.15.90:6789/0 conn(0x7ffa181454d0 legacy=0x7ffa1806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:32.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.377+0000 7ffa16000640 1 -- 172.21.15.82:0/3724273814 >> v1:172.21.15.82:6789/0 conn(0x7ffa1806e650 legacy=0x7ffa180b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.377+0000 7ffa16000640 1 -- 172.21.15.82:0/3724273814 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1815f7a0 con 0x7ffa1806e010 2024-03-16T17:58:32.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.378+0000 7ffa16000640 1 --2- 172.21.15.82:0/3724273814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1806e010 0x7ffa18160280 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815246217badba61 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:32.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.379+0000 7ffa15600640 1 -- 172.21.15.82:0/3724273814 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04031830 con 0x7ffa1806e010 2024-03-16T17:58:32.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.379+0000 7ffa15600640 1 -- 172.21.15.82:0/3724273814 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa04031990 con 0x7ffa1806e010 2024-03-16T17:58:32.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.379+0000 7ffa15600640 1 -- 172.21.15.82:0/3724273814 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04031d20 con 0x7ffa1806e010 2024-03-16T17:58:32.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.379+0000 7ffa17e00640 1 -- 172.21.15.82:0/3724273814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1806e010 msgr2=0x7ffa18160280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.379+0000 7ffa17e00640 1 --2- 172.21.15.82:0/3724273814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1806e010 0x7ffa18160280 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffa040097a0 tx=0x7ffa0402b8d0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.379+0000 7ffa17e00640 1 -- 172.21.15.82:0/3724273814 shutdown_connections 2024-03-16T17:58:32.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.379+0000 7ffa17e00640 1 --2- 172.21.15.82:0/3724273814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1806e010 0x7ffa18160280 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.379+0000 7ffa17e00640 1 -- 172.21.15.82:0/3724273814 >> 172.21.15.82:0/3724273814 conn(0x7ffa1806ae70 msgr2=0x7ffa1806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:32.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.379+0000 7ffa17e00640 1 -- 172.21.15.82:0/3724273814 shutdown_connections 2024-03-16T17:58:32.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.379+0000 7ffa17e00640 1 -- 172.21.15.82:0/3724273814 wait complete. 2024-03-16T17:58:32.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.381+0000 7ffa17e00640 1 Processor -- start 2024-03-16T17:58:32.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.381+0000 7ffa17e00640 1 -- start start 2024-03-16T17:58:32.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.382+0000 7ffa17e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa1806e650 0x7ffa180a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.382+0000 7ffa17e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffa181454d0 0x7ffa1813d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.382+0000 7ffa17e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1815afe0 0x7ffa1815d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.382+0000 7ffa17e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1815b880 con 0x7ffa1815afe0 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.382+0000 7ffa17e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa180aa570 con 0x7ffa1806e650 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.382+0000 7ffa17e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa180aa850 con 0x7ffa181454d0 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.383+0000 7ffa17400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1815afe0 0x7ffa1815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.383+0000 7ffa17400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1815afe0 0x7ffa1815d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58866/0 (socket says 172.21.15.82:58866) 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.383+0000 7ffa17400640 1 -- 172.21.15.82:0/4038897050 learned_addr learned my addr 172.21.15.82:0/4038897050 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.383+0000 7ffa17400640 1 -- 172.21.15.82:0/4038897050 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffa181454d0 msgr2=0x7ffa1813d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.383+0000 7ffa17400640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffa181454d0 0x7ffa1813d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.383+0000 7ffa17400640 1 -- 172.21.15.82:0/4038897050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa1806e650 msgr2=0x7ffa180a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.383+0000 7ffa16a00640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa1806e650 0x7ffa180a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.383+0000 7ffa17400640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa1806e650 0x7ffa180a9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.383+0000 7ffa17400640 1 -- 172.21.15.82:0/4038897050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa04009400 con 0x7ffa1815afe0 2024-03-16T17:58:32.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.383+0000 7ffa17400640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1815afe0 0x7ffa1815d750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000ca70 tx=0x7ffa1000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:32.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.384+0000 7ffa0ea00640 1 -- 172.21.15.82:0/4038897050 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1000abd0 con 0x7ffa1815afe0 2024-03-16T17:58:32.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.384+0000 7ffa0ea00640 1 -- 172.21.15.82:0/4038897050 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1000ad30 con 0x7ffa1815afe0 2024-03-16T17:58:32.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.384+0000 7ffa0ea00640 1 -- 172.21.15.82:0/4038897050 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa10012600 con 0x7ffa1815afe0 2024-03-16T17:58:32.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.384+0000 7ffa17e00640 1 -- 172.21.15.82:0/4038897050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa180ac210 con 0x7ffa1815afe0 2024-03-16T17:58:32.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.384+0000 7ffa17e00640 1 -- 172.21.15.82:0/4038897050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa180aab30 con 0x7ffa1815afe0 2024-03-16T17:58:32.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.385+0000 7ffa17e00640 1 -- 172.21.15.82:0/4038897050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa180ac070 con 0x7ffa1815afe0 2024-03-16T17:58:32.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.386+0000 7ffa0ea00640 1 -- 172.21.15.82:0/4038897050 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7ffa10007070 con 0x7ffa1815afe0 2024-03-16T17:58:32.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.387+0000 7ffa0ea00640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff9f806b3b0 0x7ff9f806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.387+0000 7ffa0ea00640 1 -- 172.21.15.82:0/4038897050 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ffa10089b20 con 0x7ffa1815afe0 2024-03-16T17:58:32.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.387+0000 7ffa16a00640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff9f806b3b0 0x7ff9f806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.387+0000 7ffa16a00640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff9f806b3b0 0x7ff9f806d870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800a750 tx=0x7ffa08008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:32.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.391+0000 7ffa0ea00640 1 -- 172.21.15.82:0/4038897050 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffa180ac070 con 0x7ffa1815afe0 2024-03-16T17:58:32.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.585+0000 7ffa17e00640 1 -- 172.21.15.82:0/4038897050 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffa180ac070 con 0x7ffa1815afe0 2024-03-16T17:58:32.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.586+0000 7ffa0ea00640 1 -- 172.21.15.82:0/4038897050 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffa180ac070 con 0x7ffa1815afe0 2024-03-16T17:58:32.590 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 -- 172.21.15.82:0/4038897050 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff9f806b3b0 msgr2=0x7ff9f806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff9f806b3b0 0x7ff9f806d870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800a750 tx=0x7ffa08008040 comp rx=0 tx=0).stop 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 -- 172.21.15.82:0/4038897050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1815afe0 msgr2=0x7ffa1815d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1815afe0 0x7ffa1815d750 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000ca70 tx=0x7ffa1000cf30 comp rx=0 tx=0).stop 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 -- 172.21.15.82:0/4038897050 shutdown_connections 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff9f806b3b0 0x7ff9f806d870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa1815afe0 0x7ffa1815d750 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffa181454d0 0x7ffa1813d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 --2- 172.21.15.82:0/4038897050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa1806e650 0x7ffa180a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 -- 172.21.15.82:0/4038897050 >> 172.21.15.82:0/4038897050 conn(0x7ffa1806ae70 msgr2=0x7ffa1813ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:32.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 -- 172.21.15.82:0/4038897050 shutdown_connections 2024-03-16T17:58:32.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.589+0000 7ffa17e00640 1 -- 172.21.15.82:0/4038897050 wait complete. 2024-03-16T17:58:32.613 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T17:58:32.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.814+0000 7f7239e00640 1 Processor -- start 2024-03-16T17:58:32.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.815+0000 7f7239e00640 1 -- start start 2024-03-16T17:58:32.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.816+0000 7f7239e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f72341616f0 0x7f72340d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.816+0000 7f7239e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7234161ad0 0x7f72340d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.816+0000 7f7239e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 0x7f72340d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.816+0000 7f7239e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72340cf8b0 con 0x7f72341616f0 2024-03-16T17:58:32.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.816+0000 7f7239e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72340b56c0 con 0x7f7234161ad0 2024-03-16T17:58:32.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.816+0000 7f7239e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72340b5100 con 0x7f72340d49a0 2024-03-16T17:58:32.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.816+0000 7f7232a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f72341616f0 0x7f72340d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.816+0000 7f7232a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f72341616f0 0x7f72340d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58880/0 (socket says 172.21.15.82:58880) 2024-03-16T17:58:32.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.816+0000 7f7232a00640 1 -- 172.21.15.82:0/3103743999 learned_addr learned my addr 172.21.15.82:0/3103743999 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:32.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.817+0000 7f7232000640 1 --2- 172.21.15.82:0/3103743999 >> v2:172.21.15.90:3300/0 conn(0x7f7234161ad0 0x7f72340d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.817+0000 7f7233400640 1 --2- 172.21.15.82:0/3103743999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 0x7f72340d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.817+0000 7f7232a00640 1 -- 172.21.15.82:0/3103743999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 msgr2=0x7f72340d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:32.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.817+0000 7f7232a00640 1 --2- 172.21.15.82:0/3103743999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 0x7f72340d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.817+0000 7f7232a00640 1 -- 172.21.15.82:0/3103743999 >> v2:172.21.15.90:3300/0 conn(0x7f7234161ad0 msgr2=0x7f72340d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:32.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.817+0000 7f7232a00640 1 --2- 172.21.15.82:0/3103743999 >> v2:172.21.15.90:3300/0 conn(0x7f7234161ad0 0x7f72340d0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.817+0000 7f7232a00640 1 -- 172.21.15.82:0/3103743999 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72340b53e0 con 0x7f72341616f0 2024-03-16T17:58:32.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.817+0000 7f7232000640 1 --2- 172.21.15.82:0/3103743999 >> v2:172.21.15.90:3300/0 conn(0x7f7234161ad0 0x7f72340d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:32.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.817+0000 7f7233400640 1 --2- 172.21.15.82:0/3103743999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 0x7f72340d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:32.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.818+0000 7f7232a00640 1 --2- 172.21.15.82:0/3103743999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72341616f0 0x7f72340d0300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f723408c960 tx=0x7f722402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf34300199d6ea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:32.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.819+0000 7f7231600640 1 -- 172.21.15.82:0/3103743999 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224031870 con 0x7f72341616f0 2024-03-16T17:58:32.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.819+0000 7f7231600640 1 -- 172.21.15.82:0/3103743999 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72240319d0 con 0x7f72341616f0 2024-03-16T17:58:32.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.819+0000 7f7231600640 1 -- 172.21.15.82:0/3103743999 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224031d60 con 0x7f72341616f0 2024-03-16T17:58:32.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.819+0000 7f7239e00640 1 -- 172.21.15.82:0/3103743999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72341616f0 msgr2=0x7f72340d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.819+0000 7f7239e00640 1 --2- 172.21.15.82:0/3103743999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72341616f0 0x7f72340d0300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f723408c960 tx=0x7f722402b910 comp rx=0 tx=0).stop 2024-03-16T17:58:32.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.819+0000 7f7239e00640 1 -- 172.21.15.82:0/3103743999 shutdown_connections 2024-03-16T17:58:32.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.819+0000 7f7239e00640 1 --2- 172.21.15.82:0/3103743999 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 0x7f72340d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.819+0000 7f7239e00640 1 --2- 172.21.15.82:0/3103743999 >> v2:172.21.15.90:3300/0 conn(0x7f7234161ad0 0x7f72340d0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.819+0000 7f7239e00640 1 --2- 172.21.15.82:0/3103743999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72341616f0 0x7f72340d0300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.819+0000 7f7239e00640 1 -- 172.21.15.82:0/3103743999 >> 172.21.15.82:0/3103743999 conn(0x7f72340d3df0 msgr2=0x7f72340d4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:32.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.820+0000 7f7239e00640 1 -- 172.21.15.82:0/3103743999 shutdown_connections 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.820+0000 7f7239e00640 1 -- 172.21.15.82:0/3103743999 wait complete. 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.821+0000 7f7239e00640 1 Processor -- start 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.821+0000 7f7239e00640 1 -- start start 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.822+0000 7f7239e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 0x7f723406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.822+0000 7f7239e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72341616f0 0x7f72340cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.822+0000 7f7239e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7234161ad0 0x7f72340b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.822+0000 7f7239e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72340b9750 con 0x7f7234161ad0 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.822+0000 7f7239e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72340b8b40 con 0x7f72341616f0 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.822+0000 7f7239e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72340b8e20 con 0x7f72340d49a0 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7233400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7234161ad0 0x7f72340b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7233400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7234161ad0 0x7f72340b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58892/0 (socket says 172.21.15.82:58892) 2024-03-16T17:58:32.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7233400640 1 -- 172.21.15.82:0/2380442503 learned_addr learned my addr 172.21.15.82:0/2380442503 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:32.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7232a00640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 0x7f723406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7232000640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72341616f0 0x7f72340cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7233400640 1 -- 172.21.15.82:0/2380442503 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 msgr2=0x7f723406cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7233400640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 0x7f723406cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7233400640 1 -- 172.21.15.82:0/2380442503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72341616f0 msgr2=0x7f72340cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7233400640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72341616f0 0x7f72340cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7233400640 1 -- 172.21.15.82:0/2380442503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72240094d0 con 0x7f7234161ad0 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7232000640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72341616f0 0x7f72340cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7232a00640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 0x7f723406cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.823+0000 7f7233400640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7234161ad0 0x7f72340b6be0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f722c00cef0 tx=0x7f722c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.824+0000 7f722b400640 1 -- 172.21.15.82:0/2380442503 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c00d840 con 0x7f7234161ad0 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.824+0000 7f722b400640 1 -- 172.21.15.82:0/2380442503 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722c00d9a0 con 0x7f7234161ad0 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.824+0000 7f722b400640 1 -- 172.21.15.82:0/2380442503 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c00dd50 con 0x7f7234161ad0 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.824+0000 7f7239e00640 1 -- 172.21.15.82:0/2380442503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72340b9100 con 0x7f7234161ad0 2024-03-16T17:58:32.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.824+0000 7f7239e00640 1 -- 172.21.15.82:0/2380442503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723406d150 con 0x7f7234161ad0 2024-03-16T17:58:32.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.825+0000 7f7239e00640 1 -- 172.21.15.82:0/2380442503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72340b30c0 con 0x7f7234161ad0 2024-03-16T17:58:32.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.826+0000 7f722b400640 1 -- 172.21.15.82:0/2380442503 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f722c01e3b0 con 0x7f7234161ad0 2024-03-16T17:58:32.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.827+0000 7f722b400640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f721006b3b0 0x7f721006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:32.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.827+0000 7f722b400640 1 -- 172.21.15.82:0/2380442503 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f722c08a770 con 0x7f7234161ad0 2024-03-16T17:58:32.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.827+0000 7f7232a00640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f721006b3b0 0x7f721006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:32.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.828+0000 7f7232a00640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f721006b3b0 0x7f721006d870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7224036000 tx=0x7f722403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:32.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:32.831+0000 7f722b400640 1 -- 172.21.15.82:0/2380442503 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f722c0552a0 con 0x7f7234161ad0 2024-03-16T17:58:33.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.036+0000 7f7239e00640 1 -- 172.21.15.82:0/2380442503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f72340b1230 con 0x7f7234161ad0 2024-03-16T17:58:33.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.036+0000 7f722b400640 1 -- 172.21.15.82:0/2380442503 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f722c058960 con 0x7f7234161ad0 2024-03-16T17:58:33.040 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 -- 172.21.15.82:0/2380442503 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f721006b3b0 msgr2=0x7f721006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f721006b3b0 0x7f721006d870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7224036000 tx=0x7f722403c040 comp rx=0 tx=0).stop 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 -- 172.21.15.82:0/2380442503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7234161ad0 msgr2=0x7f72340b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7234161ad0 0x7f72340b6be0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f722c00cef0 tx=0x7f722c00a710 comp rx=0 tx=0).stop 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 -- 172.21.15.82:0/2380442503 shutdown_connections 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f721006b3b0 0x7f721006d870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7234161ad0 0x7f72340b6be0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72341616f0 0x7f72340cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 --2- 172.21.15.82:0/2380442503 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f72340d49a0 0x7f723406cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 -- 172.21.15.82:0/2380442503 >> 172.21.15.82:0/2380442503 conn(0x7f72340d3df0 msgr2=0x7f72340d2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 -- 172.21.15.82:0/2380442503 shutdown_connections 2024-03-16T17:58:33.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.039+0000 7f7239e00640 1 -- 172.21.15.82:0/2380442503 wait complete. 2024-03-16T17:58:33.060 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-16T17:58:33.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:33.061 DEBUG:teuthology.orchestra.run.smithi090:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-16T17:58:33.163 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T17:58:33.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:33.164 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-16T17:58:33.209 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-16T17:58:33.209 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:58:33.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.423+0000 7ff32a000640 1 Processor -- start 2024-03-16T17:58:33.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.423+0000 7ff32a000640 1 -- start start 2024-03-16T17:58:33.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.425+0000 7ff32a000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3240700e0 0x7ff32415e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.425+0000 7ff32a000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff324162770 con 0x7ff3240b01a0 2024-03-16T17:58:33.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.425+0000 7ff32a000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32415e8b0 con 0x7ff32415df90 2024-03-16T17:58:33.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.425+0000 7ff32a000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240ad850 con 0x7ff3240700e0 2024-03-16T17:58:33.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.425+0000 7ff322a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff3240b01a0 0x7ff32406dad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39984/0 (socket says 172.21.15.82:39984) 2024-03-16T17:58:33.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.425+0000 7ff322a00640 1 -- 172.21.15.82:0/4215581710 learned_addr learned my addr 172.21.15.82:0/4215581710 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.425+0000 7ff323e00640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3240700e0 0x7ff32415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.425+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734363337 0 0) 0x7ff324162770 con 0x7ff3240b01a0 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.425+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff308003520 con 0x7ff3240b01a0 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4040173650 0 0) 0x7ff308003520 con 0x7ff3240b01a0 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3240700e0 msgr2=0x7ff32415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3240700e0 0x7ff32415e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 >> v1:172.21.15.90:6789/0 conn(0x7ff32415df90 legacy=0x7ff3240afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff324161b60 con 0x7ff3240b01a0 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7ff310001670 con 0x7ff3240b01a0 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 >> v1:172.21.15.82:6789/0 conn(0x7ff3240b01a0 legacy=0x7ff32406dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3080039a0 0x7ff308003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3080047c0 0x7ff308004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3240700e0 0x7ff308005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240ad850 con 0x7ff3080039a0 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30800ba30 con 0x7ff3080047c0 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff30800bbf0 con 0x7ff3240700e0 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff323400640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3080039a0 0x7ff308003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322a00640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3080047c0 0x7ff308004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff323400640 1 -- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3240700e0 msgr2=0x7ff308005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff323400640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3240700e0 0x7ff308005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff323400640 1 -- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3080047c0 msgr2=0x7ff308004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff323400640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3080047c0 0x7ff308004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff323400640 1 -- 172.21.15.82:0/4215581710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff308003520 con 0x7ff3080039a0 2024-03-16T17:58:33.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.426+0000 7ff322a00640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3080047c0 0x7ff308004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:33.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.428+0000 7ff323400640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3080039a0 0x7ff308003dc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff314001720 tx=0x7ff3080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52199caad695af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:33.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.428+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3140052d0 con 0x7ff3080039a0 2024-03-16T17:58:33.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.428+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31402e050 con 0x7ff3080039a0 2024-03-16T17:58:33.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.428+0000 7ff322000640 1 -- 172.21.15.82:0/4215581710 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3140035f0 con 0x7ff3080039a0 2024-03-16T17:58:33.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.428+0000 7ff32a000640 1 -- 172.21.15.82:0/4215581710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3080039a0 msgr2=0x7ff308003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.428+0000 7ff32a000640 1 --2- 172.21.15.82:0/4215581710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3080039a0 0x7ff308003dc0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff314001720 tx=0x7ff3080057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.428+0000 7ff323e00640 1 -- 172.21.15.82:0/4215581710 reap_dead start 2024-03-16T17:58:33.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.428+0000 7ff32a000640 1 -- 172.21.15.82:0/4215581710 shutdown_connections 2024-03-16T17:58:33.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.428+0000 7ff32a000640 1 -- 172.21.15.82:0/4215581710 >> 172.21.15.82:0/4215581710 conn(0x7ff3240d0b30 msgr2=0x7ff3240d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:33.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.429+0000 7ff32a000640 1 -- 172.21.15.82:0/4215581710 shutdown_connections 2024-03-16T17:58:33.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.429+0000 7ff32a000640 1 -- 172.21.15.82:0/4215581710 wait complete. 2024-03-16T17:58:33.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.430+0000 7ff32a000640 1 Processor -- start 2024-03-16T17:58:33.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.430+0000 7ff32a000640 1 -- start start 2024-03-16T17:58:33.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.431+0000 7ff32a000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3080039a0 0x7ff32414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.431+0000 7ff32a000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff32415b3c0 0x7ff324160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.431+0000 7ff32a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32415c250 0x7ff32415c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.431+0000 7ff32a000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32415cc40 con 0x7ff32415b3c0 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.431+0000 7ff32a000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff324161590 con 0x7ff32415c250 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.431+0000 7ff32a000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32414f950 con 0x7ff3080039a0 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.431+0000 7ff322a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff32415b3c0 0x7ff324160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff322a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff32415b3c0 0x7ff324160450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58908/0 (socket says 172.21.15.82:58908) 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff322a00640 1 -- 172.21.15.82:0/4154955384 learned_addr learned my addr 172.21.15.82:0/4154955384 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff322a00640 1 -- 172.21.15.82:0/4154955384 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3080039a0 msgr2=0x7ff32414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff323400640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3080039a0 0x7ff32414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff322a00640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3080039a0 0x7ff32414f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff322a00640 1 -- 172.21.15.82:0/4154955384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32415c250 msgr2=0x7ff32415c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff323e00640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32415c250 0x7ff32415c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff322a00640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32415c250 0x7ff32415c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff322a00640 1 -- 172.21.15.82:0/4154955384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff324160cf0 con 0x7ff32415b3c0 2024-03-16T17:58:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff322a00640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff32415b3c0 0x7ff324160450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff31000d7e0 tx=0x7ff31000dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.432+0000 7ff31b400640 1 -- 172.21.15.82:0/4154955384 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100074e0 con 0x7ff32415b3c0 2024-03-16T17:58:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.433+0000 7ff31b400640 1 -- 172.21.15.82:0/4154955384 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff310007640 con 0x7ff32415b3c0 2024-03-16T17:58:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.433+0000 7ff31b400640 1 -- 172.21.15.82:0/4154955384 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3100049e0 con 0x7ff32415b3c0 2024-03-16T17:58:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.433+0000 7ff32a000640 1 -- 172.21.15.82:0/4154955384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff324160fd0 con 0x7ff32415b3c0 2024-03-16T17:58:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.433+0000 7ff32a000640 1 -- 172.21.15.82:0/4154955384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff324150b60 con 0x7ff32415b3c0 2024-03-16T17:58:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.434+0000 7ff32a000640 1 -- 172.21.15.82:0/4154955384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff324150d20 con 0x7ff32415b3c0 2024-03-16T17:58:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.435+0000 7ff31b400640 1 -- 172.21.15.82:0/4154955384 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7ff310004b40 con 0x7ff32415b3c0 2024-03-16T17:58:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.436+0000 7ff31b400640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff30406b3e0 0x7ff30406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.436+0000 7ff31b400640 1 -- 172.21.15.82:0/4154955384 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff3100895a0 con 0x7ff32415b3c0 2024-03-16T17:58:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.436+0000 7ff323400640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff30406b3e0 0x7ff30406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.437+0000 7ff323400640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff30406b3e0 0x7ff30406d8a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff314033000 tx=0x7ff314009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.440+0000 7ff31b400640 1 -- 172.21.15.82:0/4154955384 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff310054160 con 0x7ff32415b3c0 2024-03-16T17:58:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.667+0000 7ff32a000640 1 -- 172.21.15.82:0/4154955384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff324068640 con 0x7ff32415b3c0 2024-03-16T17:58:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.669+0000 7ff31b400640 1 -- 172.21.15.82:0/4154955384 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4968 (secure 0 0 0) 0x7ff310057820 con 0x7ff32415b3c0 2024-03-16T17:58:33.673 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:33.673 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2570794110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2570794110},{"type":"v1","addr":"172.21.15.90:6835","nonce":2570794110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3135739259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3135739259},{"type":"v1","addr":"172.21.15.82:6837","nonce":3135739259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3721205060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3721205060},{"type":"v1","addr":"172.21.15.82:6835","nonce":3721205060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:57:40.797669+0000","modified":"2024-03-16T17:57:43.865338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/732041173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":732041173},{"type":"v1","addr":"172.21.15.90:6833","nonce":732041173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4267,"qdb_cluster":[4267]},"id":1}]} 2024-03-16T17:58:33.673 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-03-16T17:58:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.671+0000 7ff32a000640 1 -- 172.21.15.82:0/4154955384 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff30406b3e0 msgr2=0x7ff30406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.671+0000 7ff32a000640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff30406b3e0 0x7ff30406d8a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff314033000 tx=0x7ff314009030 comp rx=0 tx=0).stop 2024-03-16T17:58:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.671+0000 7ff32a000640 1 -- 172.21.15.82:0/4154955384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff32415b3c0 msgr2=0x7ff324160450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.671+0000 7ff32a000640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff32415b3c0 0x7ff324160450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff31000d7e0 tx=0x7ff31000dca0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.671+0000 7ff32a000640 1 -- 172.21.15.82:0/4154955384 shutdown_connections 2024-03-16T17:58:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.671+0000 7ff32a000640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff30406b3e0 0x7ff30406d8a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.671+0000 7ff32a000640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff32415c250 0x7ff32415c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.671+0000 7ff32a000640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff32415b3c0 0x7ff324160450 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.671+0000 7ff32a000640 1 --2- 172.21.15.82:0/4154955384 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3080039a0 0x7ff32414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.671+0000 7ff32a000640 1 -- 172.21.15.82:0/4154955384 >> 172.21.15.82:0/4154955384 conn(0x7ff3240d0b30 msgr2=0x7ff3240aec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.672+0000 7ff32a000640 1 -- 172.21.15.82:0/4154955384 shutdown_connections 2024-03-16T17:58:33.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.672+0000 7ff32a000640 1 -- 172.21.15.82:0/4154955384 wait complete. 2024-03-16T17:58:33.693 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:33.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.879+0000 7f0588800640 1 Processor -- start 2024-03-16T17:58:33.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.879+0000 7f0588800640 1 -- start start 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.880+0000 7f0588800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f05801454d0 0x7f05800b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.880+0000 7f0588800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f05800cdf80 0x7f05800cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.880+0000 7f0588800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05800d02b0 con 0x7f05800cfe70 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.880+0000 7f0588800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05800b4b90 con 0x7f05800cdf80 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.880+0000 7f0588800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05800cf500 con 0x7f05801454d0 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.881+0000 7f0585400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f05800cdf80 0x7f05800cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.881+0000 7f0586800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f05800cfe70 0x7f058015f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:39998/0 (socket says 172.21.15.82:39998) 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.881+0000 7f0585400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f05800cdf80 0x7f05800cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58924/0 (socket says 172.21.15.82:58924) 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.881+0000 7f0586800640 1 -- 172.21.15.82:0/399222015 learned_addr learned my addr 172.21.15.82:0/399222015 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.881+0000 7f0585e00640 1 --2- 172.21.15.82:0/399222015 >> v2:172.21.15.90:3300/0 conn(0x7f05801454d0 0x7f05800b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.881+0000 7f0585400640 1 -- 172.21.15.82:0/399222015 >> v2:172.21.15.90:3300/0 conn(0x7f05801454d0 msgr2=0x7f05800b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.881+0000 7f0585400640 1 --2- 172.21.15.82:0/399222015 >> v2:172.21.15.90:3300/0 conn(0x7f05801454d0 0x7f05800b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.881+0000 7f0585400640 1 -- 172.21.15.82:0/399222015 >> v1:172.21.15.82:6789/0 conn(0x7f05800cfe70 legacy=0x7f058015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.881+0000 7f0585400640 1 -- 172.21.15.82:0/399222015 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05801458b0 con 0x7f05800cdf80 2024-03-16T17:58:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.881+0000 7f0584a00640 1 -- 172.21.15.82:0/399222015 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583290079 0 0) 0x7f05800d02b0 con 0x7f05800cfe70 2024-03-16T17:58:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.882+0000 7f0585400640 1 --2- 172.21.15.82:0/399222015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cdf80 0x7f05800cf930 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f05800b2800 tx=0x7f057002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d25e710f2332f86 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.882+0000 7f0584a00640 1 -- 172.21.15.82:0/399222015 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570031830 con 0x7f05800cdf80 2024-03-16T17:58:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.882+0000 7f0584a00640 1 -- 172.21.15.82:0/399222015 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0570031990 con 0x7f05800cdf80 2024-03-16T17:58:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.883+0000 7f0584a00640 1 -- 172.21.15.82:0/399222015 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570031d20 con 0x7f05800cdf80 2024-03-16T17:58:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.883+0000 7f0588800640 1 -- 172.21.15.82:0/399222015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cdf80 msgr2=0x7f05800cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.883+0000 7f0588800640 1 --2- 172.21.15.82:0/399222015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cdf80 0x7f05800cf930 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f05800b2800 tx=0x7f057002b8d0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.883+0000 7f0588800640 1 -- 172.21.15.82:0/399222015 shutdown_connections 2024-03-16T17:58:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.883+0000 7f0588800640 1 --2- 172.21.15.82:0/399222015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cdf80 0x7f05800cf930 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.883+0000 7f0588800640 1 --2- 172.21.15.82:0/399222015 >> v2:172.21.15.90:3300/0 conn(0x7f05801454d0 0x7f05800b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.883+0000 7f0588800640 1 -- 172.21.15.82:0/399222015 >> 172.21.15.82:0/399222015 conn(0x7f058006ae70 msgr2=0x7f058006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.883+0000 7f0588800640 1 -- 172.21.15.82:0/399222015 shutdown_connections 2024-03-16T17:58:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.883+0000 7f0588800640 1 -- 172.21.15.82:0/399222015 wait complete. 2024-03-16T17:58:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.885+0000 7f0588800640 1 Processor -- start 2024-03-16T17:58:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.885+0000 7f0588800640 1 -- start start 2024-03-16T17:58:33.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0588800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f05800cdf80 0x7f058013e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0588800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cfe70 0x7f058006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0588800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05801454d0 0x7f058006d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0588800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800af790 con 0x7f05800cfe70 2024-03-16T17:58:33.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0588800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05800afa70 con 0x7f05801454d0 2024-03-16T17:58:33.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0588800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05800afd50 con 0x7f05800cdf80 2024-03-16T17:58:33.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0585400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cfe70 0x7f058006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0585400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cfe70 0x7f058006d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58928/0 (socket says 172.21.15.82:58928) 2024-03-16T17:58:33.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0585400640 1 -- 172.21.15.82:0/3207049464 learned_addr learned my addr 172.21.15.82:0/3207049464 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0585400640 1 -- 172.21.15.82:0/3207049464 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f05800cdf80 msgr2=0x7f058013e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0585400640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f05800cdf80 0x7f058013e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0585400640 1 -- 172.21.15.82:0/3207049464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05801454d0 msgr2=0x7f058006d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0585400640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05801454d0 0x7f058006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.886+0000 7f0585400640 1 -- 172.21.15.82:0/3207049464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0570009400 con 0x7f05800cfe70 2024-03-16T17:58:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.887+0000 7f0585400640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cfe70 0x7f058006d440 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0570009890 tx=0x7f05700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.887+0000 7f057a000640 1 -- 172.21.15.82:0/3207049464 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057003ba70 con 0x7f05800cfe70 2024-03-16T17:58:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.887+0000 7f057a000640 1 -- 172.21.15.82:0/3207049464 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057003bbf0 con 0x7f05800cfe70 2024-03-16T17:58:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.887+0000 7f0588800640 1 -- 172.21.15.82:0/3207049464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05800b0030 con 0x7f05800cfe70 2024-03-16T17:58:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.887+0000 7f057a000640 1 -- 172.21.15.82:0/3207049464 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0570045400 con 0x7f05800cfe70 2024-03-16T17:58:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.887+0000 7f0588800640 1 -- 172.21.15.82:0/3207049464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f058013e9c0 con 0x7f05800cfe70 2024-03-16T17:58:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.888+0000 7f0588800640 1 -- 172.21.15.82:0/3207049464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05800b0550 con 0x7f05800cfe70 2024-03-16T17:58:33.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.889+0000 7f057a000640 1 -- 172.21.15.82:0/3207049464 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f057004e440 con 0x7f05800cfe70 2024-03-16T17:58:33.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.890+0000 7f057a000640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f056406b3e0 0x7f056406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:33.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.890+0000 7f057a000640 1 -- 172.21.15.82:0/3207049464 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f05700abe50 con 0x7f05800cfe70 2024-03-16T17:58:33.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.890+0000 7f0585e00640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f056406b3e0 0x7f056406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:33.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.891+0000 7f0585e00640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f056406b3e0 0x7f056406d8a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0574006fd0 tx=0x7f0574008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:33.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:33.894+0000 7f057a000640 1 -- 172.21.15.82:0/3207049464 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f05800b0550 con 0x7f05800cfe70 2024-03-16T17:58:34.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.091+0000 7f0588800640 1 -- 172.21.15.82:0/3207049464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05800b0550 con 0x7f05800cfe70 2024-03-16T17:58:34.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.094+0000 7f057a000640 1 -- 172.21.15.82:0/3207049464 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16865 (secure 0 0 0) 0x7f05800b0550 con 0x7f05800cfe70 2024-03-16T17:58:34.098 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:34.098 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":23,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:27.214849+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:34.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 -- 172.21.15.82:0/3207049464 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f056406b3e0 msgr2=0x7f056406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f056406b3e0 0x7f056406d8a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0574006fd0 tx=0x7f0574008040 comp rx=0 tx=0).stop 2024-03-16T17:58:34.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 -- 172.21.15.82:0/3207049464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cfe70 msgr2=0x7f058006d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cfe70 0x7f058006d440 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0570009890 tx=0x7f05700046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 -- 172.21.15.82:0/3207049464 shutdown_connections 2024-03-16T17:58:34.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f056406b3e0 0x7f056406d8a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05801454d0 0x7f058006d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05800cfe70 0x7f058006d440 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 --2- 172.21.15.82:0/3207049464 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f05800cdf80 0x7f058013e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 -- 172.21.15.82:0/3207049464 >> 172.21.15.82:0/3207049464 conn(0x7f058006ae70 msgr2=0x7f058013f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:34.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 -- 172.21.15.82:0/3207049464 shutdown_connections 2024-03-16T17:58:34.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.096+0000 7f0588800640 1 -- 172.21.15.82:0/3207049464 wait complete. 2024-03-16T17:58:34.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:57:38.796488+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:57:39.826079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:34.120 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:34.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.321+0000 7ff900a00640 1 Processor -- start 2024-03-16T17:58:34.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.321+0000 7ff900a00640 1 -- start start 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.322+0000 7ff900a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff8fc159960 0x7ff8fc06e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.322+0000 7ff900a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff8fc06e770 0x7ff8fc0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.322+0000 7ff900a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc0cc850 0x7ff8fc15f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.322+0000 7ff900a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc159d40 con 0x7ff8fc159960 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.322+0000 7ff900a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8fc06ec20 con 0x7ff8fc06e770 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.322+0000 7ff900a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc15cff0 con 0x7ff8fc0cc850 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8f9e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff8fc159960 0x7ff8fc06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8f9e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff8fc159960 0x7ff8fc06e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58944/0 (socket says 172.21.15.82:58944) 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8f9e00640 1 -- 172.21.15.82:0/825882073 learned_addr learned my addr 172.21.15.82:0/825882073 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8f3e00640 1 --2- 172.21.15.82:0/825882073 >> v2:172.21.15.90:3300/0 conn(0x7ff8fc06e770 0x7ff8fc0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8fa800640 1 --2- 172.21.15.82:0/825882073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc0cc850 0x7ff8fc15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8f9e00640 1 -- 172.21.15.82:0/825882073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc0cc850 msgr2=0x7ff8fc15f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:34.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8f9e00640 1 --2- 172.21.15.82:0/825882073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc0cc850 0x7ff8fc15f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8f9e00640 1 -- 172.21.15.82:0/825882073 >> v2:172.21.15.90:3300/0 conn(0x7ff8fc06e770 msgr2=0x7ff8fc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:34.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8f9e00640 1 --2- 172.21.15.82:0/825882073 >> v2:172.21.15.90:3300/0 conn(0x7ff8fc06e770 0x7ff8fc0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8f9e00640 1 -- 172.21.15.82:0/825882073 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc15c370 con 0x7ff8fc159960 2024-03-16T17:58:34.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8f3e00640 1 --2- 172.21.15.82:0/825882073 >> v2:172.21.15.90:3300/0 conn(0x7ff8fc06e770 0x7ff8fc0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:34.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.323+0000 7ff8fa800640 1 --2- 172.21.15.82:0/825882073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc0cc850 0x7ff8fc15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:34.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.324+0000 7ff8f9e00640 1 --2- 172.21.15.82:0/825882073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc159960 0x7ff8fc06e230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc071dd0 tx=0x7ff8ec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda7e6a7fd94186a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:34.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.325+0000 7ff8f9400640 1 -- 172.21.15.82:0/825882073 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0317c0 con 0x7ff8fc159960 2024-03-16T17:58:34.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.325+0000 7ff8f9400640 1 -- 172.21.15.82:0/825882073 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec031920 con 0x7ff8fc159960 2024-03-16T17:58:34.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.325+0000 7ff8f9400640 1 -- 172.21.15.82:0/825882073 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec031cb0 con 0x7ff8fc159960 2024-03-16T17:58:34.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.325+0000 7ff900a00640 1 -- 172.21.15.82:0/825882073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc159960 msgr2=0x7ff8fc06e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.325+0000 7ff900a00640 1 --2- 172.21.15.82:0/825882073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc159960 0x7ff8fc06e230 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc071dd0 tx=0x7ff8ec02b860 comp rx=0 tx=0).stop 2024-03-16T17:58:34.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.326+0000 7ff900a00640 1 -- 172.21.15.82:0/825882073 shutdown_connections 2024-03-16T17:58:34.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.326+0000 7ff900a00640 1 --2- 172.21.15.82:0/825882073 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc0cc850 0x7ff8fc15f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.326+0000 7ff900a00640 1 --2- 172.21.15.82:0/825882073 >> v2:172.21.15.90:3300/0 conn(0x7ff8fc06e770 0x7ff8fc0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.326+0000 7ff900a00640 1 --2- 172.21.15.82:0/825882073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc159960 0x7ff8fc06e230 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.326+0000 7ff900a00640 1 -- 172.21.15.82:0/825882073 >> 172.21.15.82:0/825882073 conn(0x7ff8fc0b1640 msgr2=0x7ff8fc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:34.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.326+0000 7ff900a00640 1 -- 172.21.15.82:0/825882073 shutdown_connections 2024-03-16T17:58:34.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.326+0000 7ff900a00640 1 -- 172.21.15.82:0/825882073 wait complete. 2024-03-16T17:58:34.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.327+0000 7ff900a00640 1 Processor -- start 2024-03-16T17:58:34.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.327+0000 7ff900a00640 1 -- start start 2024-03-16T17:58:34.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.328+0000 7ff900a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc06e770 0x7ff8fc143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.328+0000 7ff900a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc0cc850 0x7ff8fc0c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.328+0000 7ff900a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc159960 0x7ff8fc0c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.328+0000 7ff900a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c5ca0 con 0x7ff8fc06e770 2024-03-16T17:58:34.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.328+0000 7ff900a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc0c9130 con 0x7ff8fc0cc850 2024-03-16T17:58:34.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.328+0000 7ff900a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc158bb0 con 0x7ff8fc159960 2024-03-16T17:58:34.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8f9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc06e770 0x7ff8fc143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8f9e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc06e770 0x7ff8fc143010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58956/0 (socket says 172.21.15.82:58956) 2024-03-16T17:58:34.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8f9e00640 1 -- 172.21.15.82:0/2151143286 learned_addr learned my addr 172.21.15.82:0/2151143286 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:34.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8f3e00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc0cc850 0x7ff8fc0c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8fa800640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc159960 0x7ff8fc0c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8f9e00640 1 -- 172.21.15.82:0/2151143286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc159960 msgr2=0x7ff8fc0c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8f9e00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc159960 0x7ff8fc0c5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8f9e00640 1 -- 172.21.15.82:0/2151143286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc0cc850 msgr2=0x7ff8fc0c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8f9e00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc0cc850 0x7ff8fc0c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8f9e00640 1 -- 172.21.15.82:0/2151143286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec0094d0 con 0x7ff8fc06e770 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.329+0000 7ff8f3e00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc0cc850 0x7ff8fc0c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.330+0000 7ff8f9e00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc06e770 0x7ff8fc143010 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec02b8a0 tx=0x7ff8ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.330+0000 7ff8f2000640 1 -- 172.21.15.82:0/2151143286 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec032960 con 0x7ff8fc06e770 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.330+0000 7ff8f2000640 1 -- 172.21.15.82:0/2151143286 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec032ae0 con 0x7ff8fc06e770 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.330+0000 7ff8f2000640 1 -- 172.21.15.82:0/2151143286 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec032e90 con 0x7ff8fc06e770 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.330+0000 7ff900a00640 1 -- 172.21.15.82:0/2151143286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc143550 con 0x7ff8fc06e770 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.330+0000 7ff900a00640 1 -- 172.21.15.82:0/2151143286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc0c8790 con 0x7ff8fc06e770 2024-03-16T17:58:34.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.331+0000 7ff900a00640 1 -- 172.21.15.82:0/2151143286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc0c93a0 con 0x7ff8fc06e770 2024-03-16T17:58:34.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.332+0000 7ff8f2000640 1 -- 172.21.15.82:0/2151143286 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7ff8ec04d3f0 con 0x7ff8fc06e770 2024-03-16T17:58:34.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.333+0000 7ff8f2000640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff8d806b3e0 0x7ff8d806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.333+0000 7ff8f2000640 1 -- 172.21.15.82:0/2151143286 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff8ec0abbf0 con 0x7ff8fc06e770 2024-03-16T17:58:34.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.333+0000 7ff8f3e00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff8d806b3e0 0x7ff8d806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.334+0000 7ff8f3e00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff8d806b3e0 0x7ff8d806d8a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0cdff0 tx=0x7ff8e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:34.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.338+0000 7ff8f2000640 1 -- 172.21.15.82:0/2151143286 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff8ec076830 con 0x7ff8fc06e770 2024-03-16T17:58:34.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.534+0000 7ff900a00640 1 -- 172.21.15.82:0/2151143286 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8fc067f30 con 0x7ff8fc06e770 2024-03-16T17:58:34.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.535+0000 7ff8f2000640 1 -- 172.21.15.82:0/2151143286 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8ec079ef0 con 0x7ff8fc06e770 2024-03-16T17:58:34.539 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 -- 172.21.15.82:0/2151143286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff8d806b3e0 msgr2=0x7ff8d806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff8d806b3e0 0x7ff8d806d8a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0cdff0 tx=0x7ff8e4008040 comp rx=0 tx=0).stop 2024-03-16T17:58:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 -- 172.21.15.82:0/2151143286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc06e770 msgr2=0x7ff8fc143010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc06e770 0x7ff8fc143010 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec02b8a0 tx=0x7ff8ec0046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 -- 172.21.15.82:0/2151143286 shutdown_connections 2024-03-16T17:58:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff8d806b3e0 0x7ff8d806d8a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff8fc159960 0x7ff8fc0c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff8fc0cc850 0x7ff8fc0c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 --2- 172.21.15.82:0/2151143286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8fc06e770 0x7ff8fc143010 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 -- 172.21.15.82:0/2151143286 >> 172.21.15.82:0/2151143286 conn(0x7ff8fc0b1640 msgr2=0x7ff8fc159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 -- 172.21.15.82:0/2151143286 shutdown_connections 2024-03-16T17:58:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.538+0000 7ff900a00640 1 -- 172.21.15.82:0/2151143286 wait complete. 2024-03-16T17:58:34.559 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T17:58:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.768+0000 7ff477a00640 1 Processor -- start 2024-03-16T17:58:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.768+0000 7ff477a00640 1 -- start start 2024-03-16T17:58:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.770+0000 7ff477a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff47015e760 0x7ff47015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.770+0000 7ff477a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff4700cc380 0x7ff4700cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.770+0000 7ff477a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4700ccd20 con 0x7ff470144330 2024-03-16T17:58:34.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.770+0000 7ff477a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4700ce8f0 con 0x7ff4700cc380 2024-03-16T17:58:34.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.770+0000 7ff477a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47015f2d0 con 0x7ff47015e760 2024-03-16T17:58:34.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff475000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff470144330 0x7ff470070fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:40000/0 (socket says 172.21.15.82:40000) 2024-03-16T17:58:34.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff475000640 1 -- 172.21.15.82:0/4149211055 learned_addr learned my addr 172.21.15.82:0/4149211055 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:34.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff475a00640 1 --2- 172.21.15.82:0/4149211055 >> v2:172.21.15.90:3300/0 conn(0x7ff4700cc380 0x7ff4700cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff46fe00640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff47015e760 0x7ff47015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460945097 0 0) 0x7ff4700ccd20 con 0x7ff470144330 2024-03-16T17:58:34.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff454005730 con 0x7ff470144330 2024-03-16T17:58:34.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3275806037 0 0) 0x7ff454005730 con 0x7ff470144330 2024-03-16T17:58:34.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff47015e760 msgr2=0x7ff47015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:34.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff46f400640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff47015e760 0x7ff47015ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 >> v2:172.21.15.90:3300/0 conn(0x7ff4700cc380 msgr2=0x7ff4700cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:34.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff46f400640 1 --2- 172.21.15.82:0/4149211055 >> v2:172.21.15.90:3300/0 conn(0x7ff4700cc380 0x7ff4700cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47015f5b0 con 0x7ff470144330 2024-03-16T17:58:34.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.771+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7ff4600019f0 con 0x7ff470144330 2024-03-16T17:58:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 >> v1:172.21.15.82:6789/0 conn(0x7ff470144330 legacy=0x7ff470070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff46f400640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454005b80 0x7ff454005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff46f400640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4540069d0 0x7ff454006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff46f400640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff47015e760 0x7ff454007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4700ce8f0 con 0x7ff454005b80 2024-03-16T17:58:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47015f2d0 con 0x7ff4540069d0 2024-03-16T17:58:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45400db90 con 0x7ff47015e760 2024-03-16T17:58:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff475000640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454005b80 0x7ff454005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff46fe00640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4540069d0 0x7ff454006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff475000640 1 -- 172.21.15.82:0/4149211055 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff47015e760 msgr2=0x7ff454007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff475000640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff47015e760 0x7ff454007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff475000640 1 -- 172.21.15.82:0/4149211055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4540069d0 msgr2=0x7ff454006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff475000640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4540069d0 0x7ff454006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.772+0000 7ff475000640 1 -- 172.21.15.82:0/4149211055 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff454005730 con 0x7ff454005b80 2024-03-16T17:58:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.773+0000 7ff475000640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454005b80 0x7ff454005fa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff46000a790 tx=0x7ff454007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd73ce4a260ce29f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:34.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.774+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460034630 con 0x7ff454005b80 2024-03-16T17:58:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.774+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46002f070 con 0x7ff454005b80 2024-03-16T17:58:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.774+0000 7ff46f400640 1 -- 172.21.15.82:0/4149211055 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4600349f0 con 0x7ff454005b80 2024-03-16T17:58:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.774+0000 7ff477a00640 1 -- 172.21.15.82:0/4149211055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454005b80 msgr2=0x7ff454005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.774+0000 7ff477a00640 1 --2- 172.21.15.82:0/4149211055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454005b80 0x7ff454005fa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff46000a790 tx=0x7ff454007a00 comp rx=0 tx=0).stop 2024-03-16T17:58:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.774+0000 7ff475a00640 1 -- 172.21.15.82:0/4149211055 reap_dead start 2024-03-16T17:58:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.774+0000 7ff477a00640 1 -- 172.21.15.82:0/4149211055 shutdown_connections 2024-03-16T17:58:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.774+0000 7ff477a00640 1 -- 172.21.15.82:0/4149211055 >> 172.21.15.82:0/4149211055 conn(0x7ff4700b1640 msgr2=0x7ff4700b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.774+0000 7ff477a00640 1 -- 172.21.15.82:0/4149211055 shutdown_connections 2024-03-16T17:58:34.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.774+0000 7ff477a00640 1 -- 172.21.15.82:0/4149211055 wait complete. 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.776+0000 7ff477a00640 1 Processor -- start 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.776+0000 7ff477a00640 1 -- start start 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.777+0000 7ff477a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff454005b80 0x7ff4700afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.777+0000 7ff477a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff47013d2f0 0x7ff47013d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.777+0000 7ff477a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff4701ef6a0 0x7ff4701efb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.777+0000 7ff477a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4701f0420 con 0x7ff47013d2f0 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.777+0000 7ff477a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47013cae0 con 0x7ff454005b80 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.777+0000 7ff477a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47013dcb0 con 0x7ff4701ef6a0 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.777+0000 7ff46fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff47013d2f0 0x7ff47013d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.777+0000 7ff46fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff47013d2f0 0x7ff47013d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58960/0 (socket says 172.21.15.82:58960) 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.777+0000 7ff46fe00640 1 -- 172.21.15.82:0/2198220439 learned_addr learned my addr 172.21.15.82:0/2198220439 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:34.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff475000640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff454005b80 0x7ff4700afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff46fe00640 1 -- 172.21.15.82:0/2198220439 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff4701ef6a0 msgr2=0x7ff4701efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff46fe00640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff4701ef6a0 0x7ff4701efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff46fe00640 1 -- 172.21.15.82:0/2198220439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff454005b80 msgr2=0x7ff4700afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff46fe00640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff454005b80 0x7ff4700afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff46fe00640 1 -- 172.21.15.82:0/2198220439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47013c140 con 0x7ff47013d2f0 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff475000640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff454005b80 0x7ff4700afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff46fe00640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff47013d2f0 0x7ff47013d770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff458010890 tx=0x7ff458010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff46cc00640 1 -- 172.21.15.82:0/2198220439 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458009610 con 0x7ff47013d2f0 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff46cc00640 1 -- 172.21.15.82:0/2198220439 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45800ea80 con 0x7ff47013d2f0 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff46cc00640 1 -- 172.21.15.82:0/2198220439 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800ee10 con 0x7ff47013d2f0 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.778+0000 7ff477a00640 1 -- 172.21.15.82:0/2198220439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47013c420 con 0x7ff47013d2f0 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.779+0000 7ff477a00640 1 -- 172.21.15.82:0/2198220439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47013cf00 con 0x7ff47013d2f0 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.780+0000 7ff477a00640 1 -- 172.21.15.82:0/2198220439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47013cc20 con 0x7ff47013d2f0 2024-03-16T17:58:34.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.781+0000 7ff46cc00640 1 -- 172.21.15.82:0/2198220439 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7ff45801c400 con 0x7ff47013d2f0 2024-03-16T17:58:34.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.781+0000 7ff46cc00640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff44c06b3b0 0x7ff44c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:34.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.782+0000 7ff46cc00640 1 -- 172.21.15.82:0/2198220439 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff45808a1e0 con 0x7ff47013d2f0 2024-03-16T17:58:34.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.782+0000 7ff475000640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff44c06b3b0 0x7ff44c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:34.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.782+0000 7ff475000640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff44c06b3b0 0x7ff44c06d870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff460034000 tx=0x7ff460009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:34.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.786+0000 7ff46cc00640 1 -- 172.21.15.82:0/2198220439 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff47013cc20 con 0x7ff47013d2f0 2024-03-16T17:58:34.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.981+0000 7ff477a00640 1 -- 172.21.15.82:0/2198220439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff47013cc20 con 0x7ff47013d2f0 2024-03-16T17:58:34.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.982+0000 7ff46cc00640 1 -- 172.21.15.82:0/2198220439 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff47013cc20 con 0x7ff47013d2f0 2024-03-16T17:58:34.986 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T17:58:34.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.984+0000 7ff477a00640 1 -- 172.21.15.82:0/2198220439 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff44c06b3b0 msgr2=0x7ff44c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.984+0000 7ff477a00640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff44c06b3b0 0x7ff44c06d870 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff460034000 tx=0x7ff460009030 comp rx=0 tx=0).stop 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.984+0000 7ff477a00640 1 -- 172.21.15.82:0/2198220439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff47013d2f0 msgr2=0x7ff47013d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.984+0000 7ff477a00640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff47013d2f0 0x7ff47013d770 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff458010890 tx=0x7ff458010d50 comp rx=0 tx=0).stop 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.985+0000 7ff477a00640 1 -- 172.21.15.82:0/2198220439 shutdown_connections 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.985+0000 7ff477a00640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff44c06b3b0 0x7ff44c06d870 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.985+0000 7ff477a00640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff4701ef6a0 0x7ff4701efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.985+0000 7ff477a00640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff47013d2f0 0x7ff47013d770 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.985+0000 7ff477a00640 1 --2- 172.21.15.82:0/2198220439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff454005b80 0x7ff4700afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.985+0000 7ff477a00640 1 -- 172.21.15.82:0/2198220439 >> 172.21.15.82:0/2198220439 conn(0x7ff4700b1640 msgr2=0x7ff470071980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.985+0000 7ff477a00640 1 -- 172.21.15.82:0/2198220439 shutdown_connections 2024-03-16T17:58:34.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:34.985+0000 7ff477a00640 1 -- 172.21.15.82:0/2198220439 wait complete. 2024-03-16T17:58:35.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T17:58:35.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.214+0000 7fb361800640 1 Processor -- start 2024-03-16T17:58:35.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.214+0000 7fb361800640 1 -- start start 2024-03-16T17:58:35.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.215+0000 7fb361800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb35c159960 0x7fb35c06e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.215+0000 7fb361800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35c15c370 con 0x7fb35c0cca20 2024-03-16T17:58:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.215+0000 7fb361800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35c15c650 con 0x7fb35c06e770 2024-03-16T17:58:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.215+0000 7fb361800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35c159d40 con 0x7fb35c159960 2024-03-16T17:58:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.216+0000 7fb35aa00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb35c159960 0x7fb35c06e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.216+0000 7fb35b400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb35c0cca20 0x7fb35c0cd540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35678/0 (socket says 172.21.15.82:35678) 2024-03-16T17:58:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.216+0000 7fb35b400640 1 -- 172.21.15.82:0/1212328523 learned_addr learned my addr 172.21.15.82:0/1212328523 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.216+0000 7fb35aa00640 1 -- 172.21.15.82:0/1212328523 >> v1:172.21.15.90:6789/0 conn(0x7fb35c06e770 legacy=0x7fb35c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.216+0000 7fb35aa00640 1 -- 172.21.15.82:0/1212328523 >> v1:172.21.15.82:6789/0 conn(0x7fb35c0cca20 legacy=0x7fb35c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.216+0000 7fb35aa00640 1 -- 172.21.15.82:0/1212328523 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c15c930 con 0x7fb35c159960 2024-03-16T17:58:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.217+0000 7fb35aa00640 1 --2- 172.21.15.82:0/1212328523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 0x7fb35c06e230 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0097a0 tx=0x7fb34c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03f819bf1b10d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.217+0000 7fb35a000640 1 -- 172.21.15.82:0/1212328523 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c02e070 con 0x7fb35c159960 2024-03-16T17:58:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.217+0000 7fb35a000640 1 -- 172.21.15.82:0/1212328523 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c0315a0 con 0x7fb35c159960 2024-03-16T17:58:35.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.218+0000 7fb35a000640 1 -- 172.21.15.82:0/1212328523 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c031930 con 0x7fb35c159960 2024-03-16T17:58:35.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.218+0000 7fb361800640 1 -- 172.21.15.82:0/1212328523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 msgr2=0x7fb35c06e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.218+0000 7fb361800640 1 --2- 172.21.15.82:0/1212328523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 0x7fb35c06e230 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0097a0 tx=0x7fb34c02b840 comp rx=0 tx=0).stop 2024-03-16T17:58:35.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.218+0000 7fb361800640 1 -- 172.21.15.82:0/1212328523 shutdown_connections 2024-03-16T17:58:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.218+0000 7fb361800640 1 --2- 172.21.15.82:0/1212328523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 0x7fb35c06e230 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.218+0000 7fb361800640 1 -- 172.21.15.82:0/1212328523 >> 172.21.15.82:0/1212328523 conn(0x7fb35c0b1640 msgr2=0x7fb35c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.218+0000 7fb361800640 1 -- 172.21.15.82:0/1212328523 shutdown_connections 2024-03-16T17:58:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.218+0000 7fb361800640 1 -- 172.21.15.82:0/1212328523 wait complete. 2024-03-16T17:58:35.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.220+0000 7fb361800640 1 Processor -- start 2024-03-16T17:58:35.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.220+0000 7fb361800640 1 -- start start 2024-03-16T17:58:35.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.221+0000 7fb361800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb35c06e770 0x7fb35c0b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.221+0000 7fb361800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb35c0cca20 0x7fb35c15b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.221+0000 7fb361800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 0x7fb35c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.221+0000 7fb361800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c06ec20 con 0x7fb35c159960 2024-03-16T17:58:35.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.221+0000 7fb361800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c15cff0 con 0x7fb35c06e770 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.221+0000 7fb361800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0b2a50 con 0x7fb35c0cca20 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.221+0000 7fb35b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 0x7fb35c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.221+0000 7fb35b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 0x7fb35c15b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37622/0 (socket says 172.21.15.82:37622) 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.221+0000 7fb35b400640 1 -- 172.21.15.82:0/4066397551 learned_addr learned my addr 172.21.15.82:0/4066397551 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb35aa00640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb35c06e770 0x7fb35c0b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb35b400640 1 -- 172.21.15.82:0/4066397551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb35c0cca20 msgr2=0x7fb35c15b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb35b400640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb35c0cca20 0x7fb35c15b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb35b400640 1 -- 172.21.15.82:0/4066397551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb35c06e770 msgr2=0x7fb35c0b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb35b400640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb35c06e770 0x7fb35c0b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb35b400640 1 -- 172.21.15.82:0/4066397551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c009400 con 0x7fb35c159960 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb35aa00640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb35c06e770 0x7fb35c0b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:35.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb35b400640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 0x7fb35c15b540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb35400cbc0 tx=0x7fb35400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:35.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb352a00640 1 -- 172.21.15.82:0/4066397551 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35400ad40 con 0x7fb35c159960 2024-03-16T17:58:35.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb352a00640 1 -- 172.21.15.82:0/4066397551 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb354012400 con 0x7fb35c159960 2024-03-16T17:58:35.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb352a00640 1 -- 172.21.15.82:0/4066397551 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35400aea0 con 0x7fb35c159960 2024-03-16T17:58:35.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb361800640 1 -- 172.21.15.82:0/4066397551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35c1303e0 con 0x7fb35c159960 2024-03-16T17:58:35.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.222+0000 7fb361800640 1 -- 172.21.15.82:0/4066397551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35c132cc0 con 0x7fb35c159960 2024-03-16T17:58:35.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.224+0000 7fb361800640 1 -- 172.21.15.82:0/4066397551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35c067f30 con 0x7fb35c159960 2024-03-16T17:58:35.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.225+0000 7fb352a00640 1 -- 172.21.15.82:0/4066397551 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fb354007070 con 0x7fb35c159960 2024-03-16T17:58:35.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.226+0000 7fb352a00640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb33806b3b0 0x7fb33806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.226+0000 7fb352a00640 1 -- 172.21.15.82:0/4066397551 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb354089af0 con 0x7fb35c159960 2024-03-16T17:58:35.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.226+0000 7fb35aa00640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb33806b3b0 0x7fb33806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:35.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.227+0000 7fb35aa00640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb33806b3b0 0x7fb33806d870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb34c036000 tx=0x7fb34c03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:35.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.230+0000 7fb352a00640 1 -- 172.21.15.82:0/4066397551 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb354054800 con 0x7fb35c159960 2024-03-16T17:58:35.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.426+0000 7fb361800640 1 -- 172.21.15.82:0/4066397551 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb35c130570 con 0x7fb35c159960 2024-03-16T17:58:35.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.427+0000 7fb352a00640 1 -- 172.21.15.82:0/4066397551 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb354017020 con 0x7fb35c159960 2024-03-16T17:58:35.431 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T17:58:35.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.429+0000 7fb361800640 1 -- 172.21.15.82:0/4066397551 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb33806b3b0 msgr2=0x7fb33806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.429+0000 7fb361800640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb33806b3b0 0x7fb33806d870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb34c036000 tx=0x7fb34c03d040 comp rx=0 tx=0).stop 2024-03-16T17:58:35.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.429+0000 7fb361800640 1 -- 172.21.15.82:0/4066397551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 msgr2=0x7fb35c15b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.429+0000 7fb361800640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 0x7fb35c15b540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb35400cbc0 tx=0x7fb35400a430 comp rx=0 tx=0).stop 2024-03-16T17:58:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.429+0000 7fb361800640 1 -- 172.21.15.82:0/4066397551 shutdown_connections 2024-03-16T17:58:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.429+0000 7fb361800640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb33806b3b0 0x7fb33806d870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.429+0000 7fb361800640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb35c159960 0x7fb35c15b540 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.429+0000 7fb361800640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb35c0cca20 0x7fb35c15b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.430+0000 7fb361800640 1 --2- 172.21.15.82:0/4066397551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb35c06e770 0x7fb35c0b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.430+0000 7fb361800640 1 -- 172.21.15.82:0/4066397551 >> 172.21.15.82:0/4066397551 conn(0x7fb35c0b1640 msgr2=0x7fb35c070b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.430+0000 7fb361800640 1 -- 172.21.15.82:0/4066397551 shutdown_connections 2024-03-16T17:58:35.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.430+0000 7fb361800640 1 -- 172.21.15.82:0/4066397551 wait complete. 2024-03-16T17:58:35.452 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:58:35.511 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:58:35.518 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:58:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.636+0000 7fe41c600640 1 Processor -- start 2024-03-16T17:58:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.636+0000 7fe41c600640 1 -- start start 2024-03-16T17:58:35.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.637+0000 7fe41c600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe41415e370 0x7fe4140d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.637+0000 7fe41c600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe4140d0980 0x7fe4140d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.637+0000 7fe41c600640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140cfcb0 con 0x7fe41415df90 2024-03-16T17:58:35.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.637+0000 7fe41c600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4140cf180 con 0x7fe41415e370 2024-03-16T17:58:35.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.637+0000 7fe41c600640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4140d0e20 con 0x7fe4140d0980 2024-03-16T17:58:35.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.638+0000 7fe419200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe41415e370 0x7fe4140d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:35.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.638+0000 7fe419c00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe41415df90 0x7fe4140cf5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35692/0 (socket says 172.21.15.82:35692) 2024-03-16T17:58:35.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.638+0000 7fe419c00640 1 -- 172.21.15.82:0/353315265 learned_addr learned my addr 172.21.15.82:0/353315265 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:35.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.638+0000 7fe41a600640 1 --2- 172.21.15.82:0/353315265 >> v2:172.21.15.90:3300/0 conn(0x7fe4140d0980 0x7fe4140d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:35.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.638+0000 7fe419200640 1 -- 172.21.15.82:0/353315265 >> v2:172.21.15.90:3300/0 conn(0x7fe4140d0980 msgr2=0x7fe4140d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:35.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.638+0000 7fe419200640 1 --2- 172.21.15.82:0/353315265 >> v2:172.21.15.90:3300/0 conn(0x7fe4140d0980 0x7fe4140d23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.638+0000 7fe419200640 1 -- 172.21.15.82:0/353315265 >> v1:172.21.15.82:6789/0 conn(0x7fe41415df90 legacy=0x7fe4140cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.638+0000 7fe419200640 1 -- 172.21.15.82:0/353315265 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4140ceac0 con 0x7fe41415e370 2024-03-16T17:58:35.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.638+0000 7fe413e00640 1 -- 172.21.15.82:0/353315265 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160589738 0 0) 0x7fe4140cfcb0 con 0x7fe41415df90 2024-03-16T17:58:35.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.639+0000 7fe419200640 1 --2- 172.21.15.82:0/353315265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe41415e370 0x7fe4140d0440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe414068f30 tx=0x7fe40002b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2831dec423695d2f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:35.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.639+0000 7fe413e00640 1 -- 172.21.15.82:0/353315265 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400031520 con 0x7fe41415e370 2024-03-16T17:58:35.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.640+0000 7fe413e00640 1 -- 172.21.15.82:0/353315265 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400031680 con 0x7fe41415e370 2024-03-16T17:58:35.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.640+0000 7fe413e00640 1 -- 172.21.15.82:0/353315265 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400031a10 con 0x7fe41415e370 2024-03-16T17:58:35.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.640+0000 7fe41c600640 1 -- 172.21.15.82:0/353315265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe41415e370 msgr2=0x7fe4140d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.640+0000 7fe41c600640 1 --2- 172.21.15.82:0/353315265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe41415e370 0x7fe4140d0440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe414068f30 tx=0x7fe40002b7c0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.640+0000 7fe41c600640 1 -- 172.21.15.82:0/353315265 shutdown_connections 2024-03-16T17:58:35.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.640+0000 7fe41c600640 1 --2- 172.21.15.82:0/353315265 >> v2:172.21.15.90:3300/0 conn(0x7fe4140d0980 0x7fe4140d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.640+0000 7fe41c600640 1 --2- 172.21.15.82:0/353315265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe41415e370 0x7fe4140d0440 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.640+0000 7fe41c600640 1 -- 172.21.15.82:0/353315265 >> 172.21.15.82:0/353315265 conn(0x7fe41409d0e0 msgr2=0x7fe4140d3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:35.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.640+0000 7fe41c600640 1 -- 172.21.15.82:0/353315265 shutdown_connections 2024-03-16T17:58:35.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.640+0000 7fe41c600640 1 -- 172.21.15.82:0/353315265 wait complete. 2024-03-16T17:58:35.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.642+0000 7fe41c600640 1 Processor -- start 2024-03-16T17:58:35.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.642+0000 7fe41c600640 1 -- start start 2024-03-16T17:58:35.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe41c600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4140d0980 0x7fe414152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe41c600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41415df90 0x7fe4140b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe41c600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe41415e370 0x7fe4140b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe41c600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140b6510 con 0x7fe4140d0980 2024-03-16T17:58:35.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe41c600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4141526a0 con 0x7fe41415df90 2024-03-16T17:58:35.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe41c600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4140b6fa0 con 0x7fe41415e370 2024-03-16T17:58:35.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe419c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4140d0980 0x7fe414152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:35.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe419c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4140d0980 0x7fe414152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37642/0 (socket says 172.21.15.82:37642) 2024-03-16T17:58:35.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe419c00640 1 -- 172.21.15.82:0/2390399739 learned_addr learned my addr 172.21.15.82:0/2390399739 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:35.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe419200640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41415df90 0x7fe4140b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:35.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe41a600640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe41415e370 0x7fe4140b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:35.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe419c00640 1 -- 172.21.15.82:0/2390399739 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe41415e370 msgr2=0x7fe4140b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.643+0000 7fe419c00640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe41415e370 0x7fe4140b5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.644+0000 7fe419c00640 1 -- 172.21.15.82:0/2390399739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41415df90 msgr2=0x7fe4140b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.644+0000 7fe419c00640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41415df90 0x7fe4140b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.644+0000 7fe419c00640 1 -- 172.21.15.82:0/2390399739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400009400 con 0x7fe4140d0980 2024-03-16T17:58:35.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.644+0000 7fe419200640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41415df90 0x7fe4140b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:35.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.644+0000 7fe419c00640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4140d0980 0x7fe414152160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe40800c4d0 tx=0x7fe40800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:35.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.644+0000 7fe411600640 1 -- 172.21.15.82:0/2390399739 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080074e0 con 0x7fe4140d0980 2024-03-16T17:58:35.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.644+0000 7fe411600640 1 -- 172.21.15.82:0/2390399739 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe408007660 con 0x7fe4140d0980 2024-03-16T17:58:35.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.644+0000 7fe411600640 1 -- 172.21.15.82:0/2390399739 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408004a20 con 0x7fe4140d0980 2024-03-16T17:58:35.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.644+0000 7fe41c600640 1 -- 172.21.15.82:0/2390399739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4140b7280 con 0x7fe4140d0980 2024-03-16T17:58:35.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.644+0000 7fe41c600640 1 -- 172.21.15.82:0/2390399739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4140b7840 con 0x7fe4140d0980 2024-03-16T17:58:35.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.645+0000 7fe41c600640 1 -- 172.21.15.82:0/2390399739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe414067f30 con 0x7fe4140d0980 2024-03-16T17:58:35.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.646+0000 7fe411600640 1 -- 172.21.15.82:0/2390399739 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe408004b80 con 0x7fe4140d0980 2024-03-16T17:58:35.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.647+0000 7fe411600640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3b0 0x7fe3f406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:35.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.648+0000 7fe411600640 1 -- 172.21.15.82:0/2390399739 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe408089240 con 0x7fe4140d0980 2024-03-16T17:58:35.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.648+0000 7fe419200640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3b0 0x7fe3f406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:35.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.648+0000 7fe419200640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3b0 0x7fe3f406d870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe400035000 tx=0x7fe400030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:35.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.652+0000 7fe411600640 1 -- 172.21.15.82:0/2390399739 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe408053df0 con 0x7fe4140d0980 2024-03-16T17:58:35.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.878+0000 7fe41c600640 1 -- 172.21.15.82:0/2390399739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4140d13c0 con 0x7fe4140d0980 2024-03-16T17:58:35.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.879+0000 7fe411600640 1 -- 172.21.15.82:0/2390399739 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4968 (secure 0 0 0) 0x7fe4080574b0 con 0x7fe4140d0980 2024-03-16T17:58:35.883 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:35.883 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2570794110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2570794110},{"type":"v1","addr":"172.21.15.90:6835","nonce":2570794110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3135739259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3135739259},{"type":"v1","addr":"172.21.15.82:6837","nonce":3135739259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3721205060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3721205060},{"type":"v1","addr":"172.21.15.82:6835","nonce":3721205060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:57:40.797669+0000","modified":"2024-03-16T17:57:43.865338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/732041173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":732041173},{"type":"v1","addr":"172.21.15.90:6833","nonce":732041173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4267,"qdb_cluster":[4267]},"id":1}]} 2024-03-16T17:58:35.883 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-03-16T17:58:35.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 -- 172.21.15.82:0/2390399739 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3b0 msgr2=0x7fe3f406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3b0 0x7fe3f406d870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe400035000 tx=0x7fe400030000 comp rx=0 tx=0).stop 2024-03-16T17:58:35.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 -- 172.21.15.82:0/2390399739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4140d0980 msgr2=0x7fe414152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:35.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4140d0980 0x7fe414152160 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe40800c4d0 tx=0x7fe40800c990 comp rx=0 tx=0).stop 2024-03-16T17:58:35.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 -- 172.21.15.82:0/2390399739 shutdown_connections 2024-03-16T17:58:35.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3b0 0x7fe3f406d870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe41415e370 0x7fe4140b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe41415df90 0x7fe4140b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 --2- 172.21.15.82:0/2390399739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4140d0980 0x7fe414152160 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:35.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 -- 172.21.15.82:0/2390399739 >> 172.21.15.82:0/2390399739 conn(0x7fe41409d0e0 msgr2=0x7fe4140b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:35.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 -- 172.21.15.82:0/2390399739 shutdown_connections 2024-03-16T17:58:35.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:35.882+0000 7fe41c600640 1 -- 172.21.15.82:0/2390399739 wait complete. 2024-03-16T17:58:35.901 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.123+0000 7f4ccb000640 1 Processor -- start 2024-03-16T17:58:36.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.123+0000 7f4ccb000640 1 -- start start 2024-03-16T17:58:36.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.125+0000 7f4ccb000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4cc415e180 0x7f4cc406c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.125+0000 7f4ccb000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4cc40b4430 0x7f4cc406c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.125+0000 7f4ccb000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc406d320 con 0x7f4cc406ced0 2024-03-16T17:58:36.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.125+0000 7f4ccb000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc415e560 con 0x7f4cc40b4430 2024-03-16T17:58:36.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.125+0000 7f4ccb000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc40d01b0 con 0x7f4cc415e180 2024-03-16T17:58:36.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.125+0000 7f4cc3400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4cc40b4430 0x7f4cc406c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.125+0000 7f4cc9000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4cc406ced0 0x7f4cc4161c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35696/0 (socket says 172.21.15.82:35696) 2024-03-16T17:58:36.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.125+0000 7f4cc9000640 1 -- 172.21.15.82:0/1027105949 learned_addr learned my addr 172.21.15.82:0/1027105949 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:36.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.126+0000 7f4cc3e00640 1 --2- 172.21.15.82:0/1027105949 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4cc415e180 0x7f4cc406c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.126+0000 7f4cc3400640 1 -- 172.21.15.82:0/1027105949 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4cc415e180 msgr2=0x7f4cc406c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:36.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.126+0000 7f4cc3400640 1 --2- 172.21.15.82:0/1027105949 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4cc415e180 0x7f4cc406c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.126+0000 7f4cc3400640 1 -- 172.21.15.82:0/1027105949 >> v1:172.21.15.82:6789/0 conn(0x7f4cc406ced0 legacy=0x7f4cc4161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.126+0000 7f4cc3400640 1 -- 172.21.15.82:0/1027105949 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc40d0490 con 0x7f4cc40b4430 2024-03-16T17:58:36.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.126+0000 7f4cc2a00640 1 -- 172.21.15.82:0/1027105949 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2235774700 0 0) 0x7f4cbc001490 con 0x7f4cc406ced0 2024-03-16T17:58:36.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.127+0000 7f4cc3400640 1 --2- 172.21.15.82:0/1027105949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc40b4430 0x7f4cc406c990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00097a0 tx=0x7f4cb002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ea17973fbfce06 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:36.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.127+0000 7f4cc2a00640 1 -- 172.21.15.82:0/1027105949 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0031830 con 0x7f4cc40b4430 2024-03-16T17:58:36.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.127+0000 7f4cc2a00640 1 -- 172.21.15.82:0/1027105949 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb0031990 con 0x7f4cc40b4430 2024-03-16T17:58:36.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.127+0000 7f4cc2a00640 1 -- 172.21.15.82:0/1027105949 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0031d20 con 0x7f4cc40b4430 2024-03-16T17:58:36.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.128+0000 7f4ccb000640 1 -- 172.21.15.82:0/1027105949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc40b4430 msgr2=0x7f4cc406c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.128+0000 7f4ccb000640 1 --2- 172.21.15.82:0/1027105949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc40b4430 0x7f4cc406c990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4cb00097a0 tx=0x7f4cb002b8d0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.128+0000 7f4ccb000640 1 -- 172.21.15.82:0/1027105949 shutdown_connections 2024-03-16T17:58:36.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.128+0000 7f4ccb000640 1 --2- 172.21.15.82:0/1027105949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc40b4430 0x7f4cc406c990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.128+0000 7f4ccb000640 1 --2- 172.21.15.82:0/1027105949 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4cc415e180 0x7f4cc406c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.128+0000 7f4ccb000640 1 -- 172.21.15.82:0/1027105949 >> 172.21.15.82:0/1027105949 conn(0x7f4cc40b37f0 msgr2=0x7f4cc40b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:36.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.128+0000 7f4ccb000640 1 -- 172.21.15.82:0/1027105949 shutdown_connections 2024-03-16T17:58:36.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.128+0000 7f4ccb000640 1 -- 172.21.15.82:0/1027105949 wait complete. 2024-03-16T17:58:36.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.130+0000 7f4ccb000640 1 Processor -- start 2024-03-16T17:58:36.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.130+0000 7f4ccb000640 1 -- start start 2024-03-16T17:58:36.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4ccb000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4cc406ced0 0x7f4cc415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4ccb000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cc415e180 0x7f4cc409c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4ccb000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc409cb30 0x7f4cc406ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4ccb000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc409d6f0 con 0x7f4cc409cb30 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4ccb000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc409e270 con 0x7f4cc415e180 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4ccb000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc415c500 con 0x7f4cc406ced0 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4cc9000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc409cb30 0x7f4cc406ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4cc9000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc409cb30 0x7f4cc406ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37662/0 (socket says 172.21.15.82:37662) 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4cc9000640 1 -- 172.21.15.82:0/4174935544 learned_addr learned my addr 172.21.15.82:0/4174935544 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4cc3400640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cc415e180 0x7f4cc409c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4cc3e00640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4cc406ced0 0x7f4cc415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4cc9000640 1 -- 172.21.15.82:0/4174935544 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4cc406ced0 msgr2=0x7f4cc415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4cc9000640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4cc406ced0 0x7f4cc415bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4cc9000640 1 -- 172.21.15.82:0/4174935544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cc415e180 msgr2=0x7f4cc409c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4cc9000640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cc415e180 0x7f4cc409c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.131+0000 7f4cc9000640 1 -- 172.21.15.82:0/4174935544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb0009400 con 0x7f4cc409cb30 2024-03-16T17:58:36.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.132+0000 7f4cc9000640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc409cb30 0x7f4cc406ff10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00c760 tx=0x7f4cbc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:36.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.132+0000 7f4cbbe00640 1 -- 172.21.15.82:0/4174935544 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc00ce10 con 0x7f4cc409cb30 2024-03-16T17:58:36.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.132+0000 7f4cbbe00640 1 -- 172.21.15.82:0/4174935544 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cbc00a5e0 con 0x7f4cc409cb30 2024-03-16T17:58:36.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.132+0000 7f4ccb000640 1 -- 172.21.15.82:0/4174935544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc409d940 con 0x7f4cc409cb30 2024-03-16T17:58:36.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.132+0000 7f4cbbe00640 1 -- 172.21.15.82:0/4174935544 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc00a990 con 0x7f4cc409cb30 2024-03-16T17:58:36.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.132+0000 7f4ccb000640 1 -- 172.21.15.82:0/4174935544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc409df00 con 0x7f4cc409cb30 2024-03-16T17:58:36.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.133+0000 7f4ccb000640 1 -- 172.21.15.82:0/4174935544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc4073e90 con 0x7f4cc409cb30 2024-03-16T17:58:36.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.134+0000 7f4cbbe00640 1 -- 172.21.15.82:0/4174935544 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f4cbc00aaf0 con 0x7f4cc409cb30 2024-03-16T17:58:36.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.135+0000 7f4cbbe00640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4ca406b3e0 0x7f4ca406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.135+0000 7f4cbbe00640 1 -- 172.21.15.82:0/4174935544 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4cbc089110 con 0x7f4cc409cb30 2024-03-16T17:58:36.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.135+0000 7f4cc3e00640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4ca406b3e0 0x7f4ca406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.136+0000 7f4cc3e00640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4ca406b3e0 0x7f4ca406d8a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006fd0 tx=0x7f4cb4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:36.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.140+0000 7f4cbbe00640 1 -- 172.21.15.82:0/4174935544 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4cbc053d70 con 0x7f4cc409cb30 2024-03-16T17:58:36.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.331+0000 7f4ccb000640 1 -- 172.21.15.82:0/4174935544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4cc406d320 con 0x7f4cc409cb30 2024-03-16T17:58:36.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.334+0000 7f4cbbe00640 1 -- 172.21.15.82:0/4174935544 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16865 (secure 0 0 0) 0x7f4cc406d320 con 0x7f4cc409cb30 2024-03-16T17:58:36.338 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:36.338 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":23,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:27.214849+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:36.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 -- 172.21.15.82:0/4174935544 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4ca406b3e0 msgr2=0x7f4ca406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4ca406b3e0 0x7f4ca406d8a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006fd0 tx=0x7f4cb4008040 comp rx=0 tx=0).stop 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 -- 172.21.15.82:0/4174935544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc409cb30 msgr2=0x7f4cc406ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc409cb30 0x7f4cc406ff10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00c760 tx=0x7f4cbc00cc20 comp rx=0 tx=0).stop 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 -- 172.21.15.82:0/4174935544 shutdown_connections 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f4ca406b3e0 0x7f4ca406d8a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cc409cb30 0x7f4cc406ff10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4cc415e180 0x7f4cc409c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 --2- 172.21.15.82:0/4174935544 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4cc406ced0 0x7f4cc415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 -- 172.21.15.82:0/4174935544 >> 172.21.15.82:0/4174935544 conn(0x7f4cc40b37f0 msgr2=0x7f4cc406ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 -- 172.21.15.82:0/4174935544 shutdown_connections 2024-03-16T17:58:36.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.337+0000 7f4ccb000640 1 -- 172.21.15.82:0/4174935544 wait complete. 2024-03-16T17:58:36.360 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T17:58:36.360 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:58:36.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.532+0000 7f5aeca00640 1 Processor -- start 2024-03-16T17:58:36.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.532+0000 7f5aeca00640 1 -- start start 2024-03-16T17:58:36.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.534+0000 7f5aeca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ae815e060 0x7f5ae80d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.534+0000 7f5aeca00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5ae80b2bb0 0x7f5ae80d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.534+0000 7f5aeca00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae80d2bf0 con 0x7f5ae80d27a0 2024-03-16T17:58:36.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.534+0000 7f5aeca00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae80d1de0 con 0x7f5ae815e060 2024-03-16T17:58:36.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.534+0000 7f5aeca00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae80d1b00 con 0x7f5ae80b2bb0 2024-03-16T17:58:36.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.534+0000 7f5ae6800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ae815e060 0x7f5ae80d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.534+0000 7f5ae6800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ae815e060 0x7f5ae80d0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37670/0 (socket says 172.21.15.82:37670) 2024-03-16T17:58:36.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.534+0000 7f5ae6800640 1 -- 172.21.15.82:0/3296479824 learned_addr learned my addr 172.21.15.82:0/3296479824 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:36.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.534+0000 7f5adfe00640 1 --2- 172.21.15.82:0/3296479824 >> v2:172.21.15.90:3300/0 conn(0x7f5ae80b2bb0 0x7f5ae80d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.535+0000 7f5ae6800640 1 -- 172.21.15.82:0/3296479824 >> v2:172.21.15.90:3300/0 conn(0x7f5ae80b2bb0 msgr2=0x7f5ae80d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:36.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.535+0000 7f5ae6800640 1 --2- 172.21.15.82:0/3296479824 >> v2:172.21.15.90:3300/0 conn(0x7f5ae80b2bb0 0x7f5ae80d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.535+0000 7f5ae6800640 1 -- 172.21.15.82:0/3296479824 >> v1:172.21.15.90:6789/0 conn(0x7f5ae80d27a0 legacy=0x7f5ae80ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:36.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.535+0000 7f5ae6800640 1 -- 172.21.15.82:0/3296479824 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae80d4840 con 0x7f5ae815e060 2024-03-16T17:58:36.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.535+0000 7f5adfe00640 1 --2- 172.21.15.82:0/3296479824 >> v2:172.21.15.90:3300/0 conn(0x7f5ae80b2bb0 0x7f5ae80d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:36.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.536+0000 7f5ae6800640 1 --2- 172.21.15.82:0/3296479824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 0x7f5ae80d0ce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5ae808e300 tx=0x7f5ad802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e579fb1dc0c66c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:36.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.536+0000 7f5ae5e00640 1 -- 172.21.15.82:0/3296479824 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8031870 con 0x7f5ae815e060 2024-03-16T17:58:36.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.536+0000 7f5ae5e00640 1 -- 172.21.15.82:0/3296479824 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad80319d0 con 0x7f5ae815e060 2024-03-16T17:58:36.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.536+0000 7f5ae5e00640 1 -- 172.21.15.82:0/3296479824 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8031d60 con 0x7f5ae815e060 2024-03-16T17:58:36.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.537+0000 7f5aeca00640 1 -- 172.21.15.82:0/3296479824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 msgr2=0x7f5ae80d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.537+0000 7f5aeca00640 1 --2- 172.21.15.82:0/3296479824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 0x7f5ae80d0ce0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5ae808e300 tx=0x7f5ad802b910 comp rx=0 tx=0).stop 2024-03-16T17:58:36.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.537+0000 7f5aeca00640 1 -- 172.21.15.82:0/3296479824 shutdown_connections 2024-03-16T17:58:36.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.537+0000 7f5aeca00640 1 --2- 172.21.15.82:0/3296479824 >> v2:172.21.15.90:3300/0 conn(0x7f5ae80b2bb0 0x7f5ae80d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.537+0000 7f5aeca00640 1 --2- 172.21.15.82:0/3296479824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 0x7f5ae80d0ce0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.537+0000 7f5aeca00640 1 -- 172.21.15.82:0/3296479824 >> 172.21.15.82:0/3296479824 conn(0x7f5ae80d3c80 msgr2=0x7f5ae80d4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:36.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.537+0000 7f5aeca00640 1 -- 172.21.15.82:0/3296479824 shutdown_connections 2024-03-16T17:58:36.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.537+0000 7f5aeca00640 1 -- 172.21.15.82:0/3296479824 wait complete. 2024-03-16T17:58:36.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.538+0000 7f5aeca00640 1 Processor -- start 2024-03-16T17:58:36.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.539+0000 7f5aeca00640 1 -- start start 2024-03-16T17:58:36.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.539+0000 7f5aeca00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae80b2bb0 0x7f5ae80a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.539+0000 7f5aeca00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ae80d27a0 0x7f5ae815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5aeca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 0x7f5ae80cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5aeca00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80a1230 con 0x7f5ae815e060 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5aeca00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80a1750 con 0x7f5ae80b2bb0 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5aeca00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80b7990 con 0x7f5ae80d27a0 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5ae7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 0x7f5ae80cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5ae7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 0x7f5ae80cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37686/0 (socket says 172.21.15.82:37686) 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5ae7200640 1 -- 172.21.15.82:0/2553071570 learned_addr learned my addr 172.21.15.82:0/2553071570 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5ae6800640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae80b2bb0 0x7f5ae80a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5adfe00640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ae80d27a0 0x7f5ae815c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5ae7200640 1 -- 172.21.15.82:0/2553071570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ae80d27a0 msgr2=0x7f5ae815c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5ae7200640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ae80d27a0 0x7f5ae815c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5ae7200640 1 -- 172.21.15.82:0/2553071570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae80b2bb0 msgr2=0x7f5ae80a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5ae7200640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae80b2bb0 0x7f5ae80a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5ae7200640 1 -- 172.21.15.82:0/2553071570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad80094d0 con 0x7f5ae815e060 2024-03-16T17:58:36.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5adfe00640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ae80d27a0 0x7f5ae815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:36.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.540+0000 7f5ae6800640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae80b2bb0 0x7f5ae80a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:36.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.541+0000 7f5ae7200640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 0x7f5ae80cd1f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000c7b0 tx=0x7f5ae000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:36.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.541+0000 7f5adea00640 1 -- 172.21.15.82:0/2553071570 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0004b60 con 0x7f5ae815e060 2024-03-16T17:58:36.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.541+0000 7f5adea00640 1 -- 172.21.15.82:0/2553071570 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae0004cc0 con 0x7f5ae815e060 2024-03-16T17:58:36.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.541+0000 7f5adea00640 1 -- 172.21.15.82:0/2553071570 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000aa90 con 0x7f5ae815e060 2024-03-16T17:58:36.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.541+0000 7f5aeca00640 1 -- 172.21.15.82:0/2553071570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae80b6d80 con 0x7f5ae815e060 2024-03-16T17:58:36.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.541+0000 7f5aeca00640 1 -- 172.21.15.82:0/2553071570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae80b7340 con 0x7f5ae815e060 2024-03-16T17:58:36.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.542+0000 7f5aeca00640 1 -- 172.21.15.82:0/2553071570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae804f250 con 0x7f5ae815e060 2024-03-16T17:58:36.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.543+0000 7f5adea00640 1 -- 172.21.15.82:0/2553071570 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f5ae000abf0 con 0x7f5ae815e060 2024-03-16T17:58:36.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.544+0000 7f5adea00640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5ac406b3b0 0x7f5ac406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:36.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.544+0000 7f5adea00640 1 -- 172.21.15.82:0/2553071570 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5ae0089ee0 con 0x7f5ae815e060 2024-03-16T17:58:36.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.544+0000 7f5ae6800640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5ac406b3b0 0x7f5ac406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:36.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.545+0000 7f5ae6800640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5ac406b3b0 0x7f5ac406d870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8036000 tx=0x7f5ad803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:36.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.549+0000 7f5adea00640 1 -- 172.21.15.82:0/2553071570 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5ae0054a30 con 0x7f5ae815e060 2024-03-16T17:58:36.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.779+0000 7f5aeca00640 1 -- 172.21.15.82:0/2553071570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ae80b7ad0 con 0x7f5ae815e060 2024-03-16T17:58:36.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.780+0000 7f5adea00640 1 -- 172.21.15.82:0/2553071570 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4968 (secure 0 0 0) 0x7f5ae00580f0 con 0x7f5ae815e060 2024-03-16T17:58:36.784 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:36.784 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2570794110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2570794110},{"type":"v1","addr":"172.21.15.90:6835","nonce":2570794110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3135739259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3135739259},{"type":"v1","addr":"172.21.15.82:6837","nonce":3135739259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3721205060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3721205060},{"type":"v1","addr":"172.21.15.82:6835","nonce":3721205060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:57:40.797669+0000","modified":"2024-03-16T17:57:43.865338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/732041173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":732041173},{"type":"v1","addr":"172.21.15.90:6833","nonce":732041173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4267,"qdb_cluster":[4267]},"id":1}]} 2024-03-16T17:58:36.784 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-03-16T17:58:36.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.782+0000 7f5aeca00640 1 -- 172.21.15.82:0/2553071570 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5ac406b3b0 msgr2=0x7f5ac406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.782+0000 7f5aeca00640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5ac406b3b0 0x7f5ac406d870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8036000 tx=0x7f5ad803c040 comp rx=0 tx=0).stop 2024-03-16T17:58:36.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.782+0000 7f5aeca00640 1 -- 172.21.15.82:0/2553071570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 msgr2=0x7f5ae80cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:36.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.782+0000 7f5aeca00640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 0x7f5ae80cd1f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000c7b0 tx=0x7f5ae000cc70 comp rx=0 tx=0).stop 2024-03-16T17:58:36.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.782+0000 7f5aeca00640 1 -- 172.21.15.82:0/2553071570 shutdown_connections 2024-03-16T17:58:36.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.782+0000 7f5aeca00640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5ac406b3b0 0x7f5ac406d870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.782+0000 7f5aeca00640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ae815e060 0x7f5ae80cd1f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.782+0000 7f5aeca00640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5ae80d27a0 0x7f5ae815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.782+0000 7f5aeca00640 1 --2- 172.21.15.82:0/2553071570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5ae80b2bb0 0x7f5ae80a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:36.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.782+0000 7f5aeca00640 1 -- 172.21.15.82:0/2553071570 >> 172.21.15.82:0/2553071570 conn(0x7f5ae80d3c80 msgr2=0x7f5ae80c7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:36.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.783+0000 7f5aeca00640 1 -- 172.21.15.82:0/2553071570 shutdown_connections 2024-03-16T17:58:36.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:36.783+0000 7f5aeca00640 1 -- 172.21.15.82:0/2553071570 wait complete. 2024-03-16T17:58:36.804 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:58:37.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.004+0000 7fd2dba00640 1 Processor -- start 2024-03-16T17:58:37.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.004+0000 7fd2dba00640 1 -- start start 2024-03-16T17:58:37.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.005+0000 7fd2dba00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40d00a0 0x7fd2d415b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.005+0000 7fd2dba00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d415e820 con 0x7fd2d415f550 2024-03-16T17:58:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.005+0000 7fd2dba00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d415c080 con 0x7fd2d4144400 2024-03-16T17:58:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.005+0000 7fd2dba00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d415f130 con 0x7fd2d40d00a0 2024-03-16T17:58:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d9a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd2d415f550 0x7fd2d415b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35712/0 (socket says 172.21.15.82:35712) 2024-03-16T17:58:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d9a00640 1 -- 172.21.15.82:0/1117035289 learned_addr learned my addr 172.21.15.82:0/1117035289 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129881015 0 0) 0x7fd2d415e820 con 0x7fd2d415f550 2024-03-16T17:58:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3e00640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40d00a0 0x7fd2d415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2bc005730 con 0x7fd2d415f550 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4069176476 0 0) 0x7fd2bc005730 con 0x7fd2d415f550 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40d00a0 msgr2=0x7fd2d415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40d00a0 0x7fd2d415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 >> v1:172.21.15.90:6789/0 conn(0x7fd2d4144400 legacy=0x7fd2d40cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d415ea70 con 0x7fd2d415f550 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fd2cc0019f0 con 0x7fd2d415f550 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 >> v1:172.21.15.82:6789/0 conn(0x7fd2d415f550 legacy=0x7fd2d415b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2bc005bb0 0x7fd2bc005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.006+0000 7fd2d3400640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40d00a0 0x7fd2bc0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d3400640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2bc006f10 0x7fd2bc007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d415f130 con 0x7fd2bc006f10 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc00dd70 con 0x7fd2bc005bb0 2024-03-16T17:58:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc00df30 con 0x7fd2d40d00a0 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d9a00640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2bc006f10 0x7fd2bc007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d9000640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2bc005bb0 0x7fd2bc005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d3e00640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40d00a0 0x7fd2bc0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d9a00640 1 -- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40d00a0 msgr2=0x7fd2bc0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d9a00640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40d00a0 0x7fd2bc0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d9a00640 1 -- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2bc005bb0 msgr2=0x7fd2bc005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d9a00640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2bc005bb0 0x7fd2bc005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.007+0000 7fd2d9a00640 1 -- 172.21.15.82:0/1117035289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc005730 con 0x7fd2bc006f10 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.008+0000 7fd2d9a00640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2bc006f10 0x7fd2bc007390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00a830 tx=0x7fd2bc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c4ce7c6e8ff6d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.009+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0072d0 con 0x7fd2bc006f10 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.009+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc02f070 con 0x7fd2bc006f10 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.009+0000 7fd2d3400640 1 -- 172.21.15.82:0/1117035289 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0076b0 con 0x7fd2bc006f10 2024-03-16T17:58:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.009+0000 7fd2dba00640 1 -- 172.21.15.82:0/1117035289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2bc006f10 msgr2=0x7fd2bc007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.009+0000 7fd2dba00640 1 --2- 172.21.15.82:0/1117035289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2bc006f10 0x7fd2bc007390 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc00a830 tx=0x7fd2bc007a00 comp rx=0 tx=0).stop 2024-03-16T17:58:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.009+0000 7fd2d9a00640 1 -- 172.21.15.82:0/1117035289 reap_dead start 2024-03-16T17:58:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.009+0000 7fd2dba00640 1 -- 172.21.15.82:0/1117035289 shutdown_connections 2024-03-16T17:58:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.009+0000 7fd2dba00640 1 -- 172.21.15.82:0/1117035289 >> 172.21.15.82:0/1117035289 conn(0x7fd2d406ae70 msgr2=0x7fd2d406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.009+0000 7fd2dba00640 1 -- 172.21.15.82:0/1117035289 shutdown_connections 2024-03-16T17:58:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.009+0000 7fd2dba00640 1 -- 172.21.15.82:0/1117035289 wait complete. 2024-03-16T17:58:37.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.011+0000 7fd2dba00640 1 Processor -- start 2024-03-16T17:58:37.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.011+0000 7fd2dba00640 1 -- start start 2024-03-16T17:58:37.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2dba00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2bc006f10 0x7fd2d41eed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2dba00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40b4b20 0x7fd2d40b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2dba00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d40b05b0 0x7fd2d4141860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2dba00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d413ba80 con 0x7fd2d40b05b0 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2dba00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d413ae70 con 0x7fd2bc006f10 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2dba00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d413b150 con 0x7fd2d40b4b20 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2d9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d40b05b0 0x7fd2d4141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2d9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d40b05b0 0x7fd2d4141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37704/0 (socket says 172.21.15.82:37704) 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2d9a00640 1 -- 172.21.15.82:0/3490085848 learned_addr learned my addr 172.21.15.82:0/3490085848 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2d9000640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2bc006f10 0x7fd2d41eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.012+0000 7fd2d3e00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40b4b20 0x7fd2d40b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2d9a00640 1 -- 172.21.15.82:0/3490085848 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40b4b20 msgr2=0x7fd2d40b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2d9a00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40b4b20 0x7fd2d40b0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2d9a00640 1 -- 172.21.15.82:0/3490085848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2bc006f10 msgr2=0x7fd2d41eed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2d9a00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2bc006f10 0x7fd2d41eed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2d9a00640 1 -- 172.21.15.82:0/3490085848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d413b430 con 0x7fd2d40b05b0 2024-03-16T17:58:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2d9a00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d40b05b0 0x7fd2d4141860 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc001cb0 tx=0x7fd2cc001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2d0c00640 1 -- 172.21.15.82:0/3490085848 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0074e0 con 0x7fd2d40b05b0 2024-03-16T17:58:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2d0c00640 1 -- 172.21.15.82:0/3490085848 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc007640 con 0x7fd2d40b05b0 2024-03-16T17:58:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2dba00640 1 -- 172.21.15.82:0/3490085848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d413b710 con 0x7fd2d40b05b0 2024-03-16T17:58:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2d0c00640 1 -- 172.21.15.82:0/3490085848 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc0079d0 con 0x7fd2d40b05b0 2024-03-16T17:58:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.013+0000 7fd2dba00640 1 -- 172.21.15.82:0/3490085848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d413aa10 con 0x7fd2d40b05b0 2024-03-16T17:58:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.015+0000 7fd2dba00640 1 -- 172.21.15.82:0/3490085848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d413b8a0 con 0x7fd2d40b05b0 2024-03-16T17:58:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.015+0000 7fd2d0c00640 1 -- 172.21.15.82:0/3490085848 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fd2cc031030 con 0x7fd2d40b05b0 2024-03-16T17:58:37.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.017+0000 7fd2d0c00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd2b406b3b0 0x7fd2b406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.017+0000 7fd2d0c00640 1 -- 172.21.15.82:0/3490085848 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd2cc0ac280 con 0x7fd2d40b05b0 2024-03-16T17:58:37.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.017+0000 7fd2d9000640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd2b406b3b0 0x7fd2b406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.017+0000 7fd2d9000640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd2b406b3b0 0x7fd2b406d870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80017e0 tx=0x7fd2c8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:37.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.021+0000 7fd2d0c00640 1 -- 172.21.15.82:0/3490085848 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd2cc076ed0 con 0x7fd2d40b05b0 2024-03-16T17:58:37.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.236+0000 7fd2dba00640 1 -- 172.21.15.82:0/3490085848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2d413b8a0 con 0x7fd2d40b05b0 2024-03-16T17:58:37.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.237+0000 7fd2d0c00640 1 -- 172.21.15.82:0/3490085848 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4968 (secure 0 0 0) 0x7fd2cc07a590 con 0x7fd2d40b05b0 2024-03-16T17:58:37.241 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:37.241 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4202,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2570794110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2570794110},{"type":"v1","addr":"172.21.15.90:6835","nonce":2570794110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3135739259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3135739259},{"type":"v1","addr":"172.21.15.82:6837","nonce":3135739259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3721205060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3721205060},{"type":"v1","addr":"172.21.15.82:6835","nonce":3721205060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:57:40.797669+0000","modified":"2024-03-16T17:57:43.865338+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.90:6833/732041173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":732041173},{"type":"v1","addr":"172.21.15.90:6833","nonce":732041173}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4267,"qdb_cluster":[4267]},"id":1}]} 2024-03-16T17:58:37.241 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-03-16T17:58:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.239+0000 7fd2dba00640 1 -- 172.21.15.82:0/3490085848 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd2b406b3b0 msgr2=0x7fd2b406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.239+0000 7fd2dba00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd2b406b3b0 0x7fd2b406d870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80017e0 tx=0x7fd2c8009040 comp rx=0 tx=0).stop 2024-03-16T17:58:37.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.239+0000 7fd2dba00640 1 -- 172.21.15.82:0/3490085848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d40b05b0 msgr2=0x7fd2d4141860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.239+0000 7fd2dba00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d40b05b0 0x7fd2d4141860 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc001cb0 tx=0x7fd2cc001d80 comp rx=0 tx=0).stop 2024-03-16T17:58:37.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.240+0000 7fd2dba00640 1 -- 172.21.15.82:0/3490085848 shutdown_connections 2024-03-16T17:58:37.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.240+0000 7fd2dba00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd2b406b3b0 0x7fd2b406d870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.240+0000 7fd2dba00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2d40b05b0 0x7fd2d4141860 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.240+0000 7fd2dba00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd2d40b4b20 0x7fd2d40b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.240+0000 7fd2dba00640 1 --2- 172.21.15.82:0/3490085848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2bc006f10 0x7fd2d41eed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.240+0000 7fd2dba00640 1 -- 172.21.15.82:0/3490085848 >> 172.21.15.82:0/3490085848 conn(0x7fd2d406ae70 msgr2=0x7fd2d415a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:37.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.240+0000 7fd2dba00640 1 -- 172.21.15.82:0/3490085848 shutdown_connections 2024-03-16T17:58:37.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.240+0000 7fd2dba00640 1 -- 172.21.15.82:0/3490085848 wait complete. 2024-03-16T17:58:37.261 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:37.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.463+0000 7f2939c00640 1 Processor -- start 2024-03-16T17:58:37.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.463+0000 7f2939c00640 1 -- start start 2024-03-16T17:58:37.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.465+0000 7f2939c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29340b2ce0 0x7f29340af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.465+0000 7f2939c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29340b1840 con 0x7f29340afb90 2024-03-16T17:58:37.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.465+0000 7f2939c00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29340b0950 con 0x7f29340af270 2024-03-16T17:58:37.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.465+0000 7f2939c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29340b0c30 con 0x7f29340b2ce0 2024-03-16T17:58:37.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.465+0000 7f2933400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f29340afb90 0x7f29341659f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35716/0 (socket says 172.21.15.82:35716) 2024-03-16T17:58:37.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.465+0000 7f2933400640 1 -- 172.21.15.82:0/1020985446 learned_addr learned my addr 172.21.15.82:0/1020985446 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:37.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2932000640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29340b2ce0 0x7f29340af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321805072 0 0) 0x7f29340b1840 con 0x7f29340afb90 2024-03-16T17:58:37.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2914003520 con 0x7f29340afb90 2024-03-16T17:58:37.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2190875886 0 0) 0x7f2914003520 con 0x7f29340afb90 2024-03-16T17:58:37.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29340b2ce0 msgr2=0x7f29340af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:37.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29340b2ce0 0x7f29340af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 >> v1:172.21.15.90:6789/0 conn(0x7f29340af270 legacy=0x7f29340b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29340b0f10 con 0x7f29340afb90 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f292c001670 con 0x7f29340afb90 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 >> v1:172.21.15.82:6789/0 conn(0x7f29340afb90 legacy=0x7f29341659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29340b2ce0 0x7f29140039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2914003f20 0x7f29140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.466+0000 7f2931600640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2914004d00 0x7f2914005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29340b0c30 con 0x7f2914004d00 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291400baf0 con 0x7f2914003f20 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291400bcb0 con 0x7f29340b2ce0 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2933400640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2914004d00 0x7f2914005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2932000640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2914003f20 0x7f29140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2932a00640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29340b2ce0 0x7f29140039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2933400640 1 -- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29340b2ce0 msgr2=0x7f29140039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2933400640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f29340b2ce0 0x7f29140039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2933400640 1 -- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2914003f20 msgr2=0x7f29140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:37.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2933400640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2914003f20 0x7f29140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.467+0000 7f2933400640 1 -- 172.21.15.82:0/1020985446 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2914003520 con 0x7f2914004d00 2024-03-16T17:58:37.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.468+0000 7f2933400640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2914004d00 0x7f2914005180 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c240 tx=0x7f29140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86867ae3b3c445cb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:37.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.469+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c005ad0 con 0x7f2914004d00 2024-03-16T17:58:37.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.469+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292c005c30 con 0x7f2914004d00 2024-03-16T17:58:37.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.469+0000 7f2931600640 1 -- 172.21.15.82:0/1020985446 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292c02caf0 con 0x7f2914004d00 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.469+0000 7f2939c00640 1 -- 172.21.15.82:0/1020985446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2914004d00 msgr2=0x7f2914005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.469+0000 7f2939c00640 1 --2- 172.21.15.82:0/1020985446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2914004d00 0x7f2914005180 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c240 tx=0x7f29140057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.469+0000 7f2933400640 1 -- 172.21.15.82:0/1020985446 reap_dead start 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.469+0000 7f2939c00640 1 -- 172.21.15.82:0/1020985446 shutdown_connections 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.469+0000 7f2939c00640 1 -- 172.21.15.82:0/1020985446 >> 172.21.15.82:0/1020985446 conn(0x7f293409a9e0 msgr2=0x7f293409adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.469+0000 7f2939c00640 1 -- 172.21.15.82:0/1020985446 shutdown_connections 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.469+0000 7f2939c00640 1 -- 172.21.15.82:0/1020985446 wait complete. 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.471+0000 7f2939c00640 1 Processor -- start 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.471+0000 7f2939c00640 1 -- start start 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.472+0000 7f2939c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2914004d00 0x7f2934151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.472+0000 7f2939c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2934150b60 0x7f2934153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.472+0000 7f2939c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f293415f0d0 0x7f293415ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.472+0000 7f2939c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2934153780 con 0x7f2934150b60 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.472+0000 7f2939c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29341552d0 con 0x7f293415f0d0 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.472+0000 7f2939c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29341540e0 con 0x7f2914004d00 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.472+0000 7f2932000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2934150b60 0x7f2934153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.472+0000 7f2932000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2934150b60 0x7f2934153ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37720/0 (socket says 172.21.15.82:37720) 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.472+0000 7f2932000640 1 -- 172.21.15.82:0/650247721 learned_addr learned my addr 172.21.15.82:0/650247721 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2933400640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f293415f0d0 0x7f293415ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2932a00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2914004d00 0x7f2934151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2932000640 1 -- 172.21.15.82:0/650247721 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2914004d00 msgr2=0x7f2934151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2932000640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2914004d00 0x7f2934151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2932000640 1 -- 172.21.15.82:0/650247721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f293415f0d0 msgr2=0x7f293415ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2932000640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f293415f0d0 0x7f293415ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2932000640 1 -- 172.21.15.82:0/650247721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2934154ff0 con 0x7f2934150b60 2024-03-16T17:58:37.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2932a00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2914004d00 0x7f2934151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:37.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2932000640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2934150b60 0x7f2934153ba0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f291c007f30 tx=0x7f291c011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:37.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f292aa00640 1 -- 172.21.15.82:0/650247721 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c00e030 con 0x7f2934150b60 2024-03-16T17:58:37.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f292aa00640 1 -- 172.21.15.82:0/650247721 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c016470 con 0x7f2934150b60 2024-03-16T17:58:37.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f292aa00640 1 -- 172.21.15.82:0/650247721 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c016820 con 0x7f2934150b60 2024-03-16T17:58:37.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2939c00640 1 -- 172.21.15.82:0/650247721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29341530c0 con 0x7f2934150b60 2024-03-16T17:58:37.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.473+0000 7f2939c00640 1 -- 172.21.15.82:0/650247721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2934154c30 con 0x7f2934150b60 2024-03-16T17:58:37.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.475+0000 7f2939c00640 1 -- 172.21.15.82:0/650247721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2934152f20 con 0x7f2934150b60 2024-03-16T17:58:37.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.475+0000 7f292aa00640 1 -- 172.21.15.82:0/650247721 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f291c016980 con 0x7f2934150b60 2024-03-16T17:58:37.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.476+0000 7f292aa00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f291006b3e0 0x7f291006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.476+0000 7f292aa00640 1 -- 172.21.15.82:0/650247721 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f291c00f070 con 0x7f2934150b60 2024-03-16T17:58:37.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.476+0000 7f2932a00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f291006b3e0 0x7f291006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.477+0000 7f2932a00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f291006b3e0 0x7f291006d8a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f293415ba40 tx=0x7f2924009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:37.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.480+0000 7f292aa00640 1 -- 172.21.15.82:0/650247721 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2934152f20 con 0x7f2934150b60 2024-03-16T17:58:37.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.677+0000 7f2939c00640 1 -- 172.21.15.82:0/650247721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2934152f20 con 0x7f2934150b60 2024-03-16T17:58:37.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.679+0000 7f292aa00640 1 -- 172.21.15.82:0/650247721 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16865 (secure 0 0 0) 0x7f2934152f20 con 0x7f2934150b60 2024-03-16T17:58:37.683 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:37.683 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":23,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:27.214849+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:57:38.796488+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T17:57:39.826079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.1f","mappings":[{"from":0,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":2}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:37.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.681+0000 7f2939c00640 1 -- 172.21.15.82:0/650247721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f291006b3e0 msgr2=0x7f291006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.681+0000 7f2939c00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f291006b3e0 0x7f291006d8a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f293415ba40 tx=0x7f2924009040 comp rx=0 tx=0).stop 2024-03-16T17:58:37.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.682+0000 7f2939c00640 1 -- 172.21.15.82:0/650247721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2934150b60 msgr2=0x7f2934153ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.682+0000 7f2939c00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2934150b60 0x7f2934153ba0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f291c007f30 tx=0x7f291c011660 comp rx=0 tx=0).stop 2024-03-16T17:58:37.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.682+0000 7f2939c00640 1 -- 172.21.15.82:0/650247721 shutdown_connections 2024-03-16T17:58:37.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.682+0000 7f2939c00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f291006b3e0 0x7f291006d8a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.682+0000 7f2939c00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f293415f0d0 0x7f293415ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.682+0000 7f2939c00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2934150b60 0x7f2934153ba0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.682+0000 7f2939c00640 1 --2- 172.21.15.82:0/650247721 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2914004d00 0x7f2934151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.682+0000 7f2939c00640 1 -- 172.21.15.82:0/650247721 >> 172.21.15.82:0/650247721 conn(0x7f293409a9e0 msgr2=0x7f293406f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:37.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.682+0000 7f2939c00640 1 -- 172.21.15.82:0/650247721 shutdown_connections 2024-03-16T17:58:37.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.682+0000 7f2939c00640 1 -- 172.21.15.82:0/650247721 wait complete. 2024-03-16T17:58:37.703 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-16T17:58:37.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.901+0000 7fc5af200640 1 Processor -- start 2024-03-16T17:58:37.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.902+0000 7fc5af200640 1 -- start start 2024-03-16T17:58:37.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.903+0000 7fc5af200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5b00d04f0 0x7fc5b00b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.903+0000 7fc5af200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00d0fe0 0x7fc5b00b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.903+0000 7fc5af200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b00d42d0 con 0x7fc5b00cfad0 2024-03-16T17:58:37.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.903+0000 7fc5af200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b00d1490 con 0x7fc5b00d04f0 2024-03-16T17:58:37.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.903+0000 7fc5af200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b00cff50 con 0x7fc5b00d0fe0 2024-03-16T17:58:37.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.903+0000 7fc5ade00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5b00d04f0 0x7fc5b00b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.903+0000 7fc5ade00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5b00d04f0 0x7fc5b00b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37732/0 (socket says 172.21.15.82:37732) 2024-03-16T17:58:37.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.903+0000 7fc5ade00640 1 -- 172.21.15.82:0/4028677338 learned_addr learned my addr 172.21.15.82:0/4028677338 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:37.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.904+0000 7fc5ae800640 1 --2- 172.21.15.82:0/4028677338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00d0fe0 0x7fc5b00b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.904+0000 7fc5ade00640 1 -- 172.21.15.82:0/4028677338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00d0fe0 msgr2=0x7fc5b00b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:37.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.904+0000 7fc5ade00640 1 --2- 172.21.15.82:0/4028677338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00d0fe0 0x7fc5b00b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.904+0000 7fc5ade00640 1 -- 172.21.15.82:0/4028677338 >> v1:172.21.15.90:6789/0 conn(0x7fc5b00cfad0 legacy=0x7fc5b00d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:37.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.904+0000 7fc5ade00640 1 -- 172.21.15.82:0/4028677338 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b015e6e0 con 0x7fc5b00d04f0 2024-03-16T17:58:37.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.905+0000 7fc5ade00640 1 --2- 172.21.15.82:0/4028677338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 0x7fc5b00b5050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009e50 tx=0x7fc5a002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bc9858407b1312 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:37.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.905+0000 7fc5ad400640 1 -- 172.21.15.82:0/4028677338 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a002ca80 con 0x7fc5b00d04f0 2024-03-16T17:58:37.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.905+0000 7fc5ad400640 1 -- 172.21.15.82:0/4028677338 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a002cbe0 con 0x7fc5b00d04f0 2024-03-16T17:58:37.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.905+0000 7fc5ad400640 1 -- 172.21.15.82:0/4028677338 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00343f0 con 0x7fc5b00d04f0 2024-03-16T17:58:37.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.906+0000 7fc5af200640 1 -- 172.21.15.82:0/4028677338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 msgr2=0x7fc5b00b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.906+0000 7fc5af200640 1 --2- 172.21.15.82:0/4028677338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 0x7fc5b00b5050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009e50 tx=0x7fc5a002bee0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.906+0000 7fc5af200640 1 -- 172.21.15.82:0/4028677338 shutdown_connections 2024-03-16T17:58:37.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.906+0000 7fc5af200640 1 --2- 172.21.15.82:0/4028677338 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00d0fe0 0x7fc5b00b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.906+0000 7fc5af200640 1 --2- 172.21.15.82:0/4028677338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 0x7fc5b00b5050 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.906+0000 7fc5af200640 1 -- 172.21.15.82:0/4028677338 >> 172.21.15.82:0/4028677338 conn(0x7fc5b009cf30 msgr2=0x7fc5b00d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:37.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.906+0000 7fc5af200640 1 -- 172.21.15.82:0/4028677338 shutdown_connections 2024-03-16T17:58:37.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.906+0000 7fc5af200640 1 -- 172.21.15.82:0/4028677338 wait complete. 2024-03-16T17:58:37.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.907+0000 7fc5af200640 1 Processor -- start 2024-03-16T17:58:37.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.908+0000 7fc5af200640 1 -- start start 2024-03-16T17:58:37.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.908+0000 7fc5af200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00cfad0 0x7fc5b01510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.908+0000 7fc5af200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 0x7fc5b00cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.908+0000 7fc5af200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b00d0fe0 0x7fc5b00b9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.908+0000 7fc5af200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b015b490 con 0x7fc5b00d04f0 2024-03-16T17:58:37.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.908+0000 7fc5af200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b01515e0 con 0x7fc5b00d0fe0 2024-03-16T17:58:37.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.908+0000 7fc5af200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b00b7fd0 con 0x7fc5b00cfad0 2024-03-16T17:58:37.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5a5400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 0x7fc5b00cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5a5400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 0x7fc5b00cd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37734/0 (socket says 172.21.15.82:37734) 2024-03-16T17:58:37.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5a5400640 1 -- 172.21.15.82:0/2001572693 learned_addr learned my addr 172.21.15.82:0/2001572693 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:37.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5ae800640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b00d0fe0 0x7fc5b00b9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5ade00640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00cfad0 0x7fc5b01510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5a5400640 1 -- 172.21.15.82:0/2001572693 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00cfad0 msgr2=0x7fc5b01510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5a5400640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00cfad0 0x7fc5b01510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5a5400640 1 -- 172.21.15.82:0/2001572693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b00d0fe0 msgr2=0x7fc5b00b9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:37.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5a5400640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b00d0fe0 0x7fc5b00b9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:37.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5a5400640 1 -- 172.21.15.82:0/2001572693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0009b30 con 0x7fc5b00d04f0 2024-03-16T17:58:37.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5ade00640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00cfad0 0x7fc5b01510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:37.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.909+0000 7fc5a5400640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 0x7fc5b00cd890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc59800b3c0 tx=0x7fc59800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:37.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.910+0000 7fc5a6a00640 1 -- 172.21.15.82:0/2001572693 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598011830 con 0x7fc5b00d04f0 2024-03-16T17:58:37.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.910+0000 7fc5a6a00640 1 -- 172.21.15.82:0/2001572693 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598011990 con 0x7fc5b00d04f0 2024-03-16T17:58:37.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.910+0000 7fc5a6a00640 1 -- 172.21.15.82:0/2001572693 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598011d80 con 0x7fc5b00d04f0 2024-03-16T17:58:37.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.910+0000 7fc5af200640 1 -- 172.21.15.82:0/2001572693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b00b82b0 con 0x7fc5b00d04f0 2024-03-16T17:58:37.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.910+0000 7fc5af200640 1 -- 172.21.15.82:0/2001572693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b00b8870 con 0x7fc5b00d04f0 2024-03-16T17:58:37.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.911+0000 7fc5af200640 1 -- 172.21.15.82:0/2001572693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b00d15e0 con 0x7fc5b00d04f0 2024-03-16T17:58:37.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.912+0000 7fc5a6a00640 1 -- 172.21.15.82:0/2001572693 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fc59801b460 con 0x7fc5b00d04f0 2024-03-16T17:58:37.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.913+0000 7fc5a6a00640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc58c06b3b0 0x7fc58c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:37.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.913+0000 7fc5a6a00640 1 -- 172.21.15.82:0/2001572693 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fc5980898d0 con 0x7fc5b00d04f0 2024-03-16T17:58:37.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.914+0000 7fc5ade00640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc58c06b3b0 0x7fc58c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:37.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.914+0000 7fc5ade00640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc58c06b3b0 0x7fc58c06d870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0035000 tx=0x7fc5a0030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:37.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:37.917+0000 7fc5a6a00640 1 -- 172.21.15.82:0/2001572693 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc598054400 con 0x7fc5b00d04f0 2024-03-16T17:58:38.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.137+0000 7fc5af200640 1 -- 172.21.15.82:0/2001572693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc5b004f250 con 0x7fc5b00d04f0 2024-03-16T17:58:38.141 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-03-16T17:58:38.138+0000 7f1de0600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T17:58:38.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.142+0000 7fc5a6a00640 1 -- 172.21.15.82:0/2001572693 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc598057ac0 con 0x7fc5b00d04f0 2024-03-16T17:58:38.145 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-16T17:58:38.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.144+0000 7fc5af200640 1 -- 172.21.15.82:0/2001572693 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc58c06b3b0 msgr2=0x7fc58c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:38.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.144+0000 7fc5af200640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc58c06b3b0 0x7fc58c06d870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0035000 tx=0x7fc5a0030000 comp rx=0 tx=0).stop 2024-03-16T17:58:38.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.144+0000 7fc5af200640 1 -- 172.21.15.82:0/2001572693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 msgr2=0x7fc5b00cd890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:38.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.144+0000 7fc5af200640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 0x7fc5b00cd890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc59800b3c0 tx=0x7fc59800b880 comp rx=0 tx=0).stop 2024-03-16T17:58:38.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.145+0000 7fc5af200640 1 -- 172.21.15.82:0/2001572693 shutdown_connections 2024-03-16T17:58:38.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.145+0000 7fc5af200640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc58c06b3b0 0x7fc58c06d870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:38.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.145+0000 7fc5af200640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5b00d0fe0 0x7fc5b00b9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:38.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.145+0000 7fc5af200640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5b00d04f0 0x7fc5b00cd890 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:38.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.145+0000 7fc5af200640 1 --2- 172.21.15.82:0/2001572693 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc5b00cfad0 0x7fc5b01510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:38.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.145+0000 7fc5af200640 1 -- 172.21.15.82:0/2001572693 >> 172.21.15.82:0/2001572693 conn(0x7fc5b009cf30 msgr2=0x7fc5b00b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:38.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.145+0000 7fc5af200640 1 -- 172.21.15.82:0/2001572693 shutdown_connections 2024-03-16T17:58:38.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.145+0000 7fc5af200640 1 -- 172.21.15.82:0/2001572693 wait complete. 2024-03-16T17:58:38.167 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-16T17:58:38.187 INFO:tasks.ceph.mds.b.smithi090.stderr: -7743> 2024-03-16T17:57:44.674+0000 7f9bb4800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T17:58:38.269 INFO:tasks.ceph.mds.b.smithi090.stdout:starting mds.b at 2024-03-16T17:58:38.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.365+0000 7fab1d800640 1 Processor -- start 2024-03-16T17:58:38.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.366+0000 7fab1d800640 1 -- start start 2024-03-16T17:58:38.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.367+0000 7fab1d800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 0x7fab18162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:38.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.367+0000 7fab1d800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fab180cddc0 0x7fab180ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:38.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.367+0000 7fab1d800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab18146da0 con 0x7fab18147060 2024-03-16T17:58:38.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.367+0000 7fab1d800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab180ce7b0 con 0x7fab180cddc0 2024-03-16T17:58:38.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.367+0000 7fab1d800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab181628a0 con 0x7fab181469c0 2024-03-16T17:58:38.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.368+0000 7fab17400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fab180cddc0 0x7fab180ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:38.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.368+0000 7fab16a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 0x7fab18162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:38.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.368+0000 7fab16000640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fab18147060 0x7fab1806e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.82:47612/0 (socket says 172.21.15.82:47612) 2024-03-16T17:58:38.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.368+0000 7fab16000640 1 -- 172.21.15.82:0/151884146 learned_addr learned my addr 172.21.15.82:0/151884146 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:38.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.369+0000 7fab16a00640 1 -- 172.21.15.82:0/151884146 >> v2:172.21.15.90:3300/0 conn(0x7fab180cddc0 msgr2=0x7fab180ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:38.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.369+0000 7fab16a00640 1 --2- 172.21.15.82:0/151884146 >> v2:172.21.15.90:3300/0 conn(0x7fab180cddc0 0x7fab180ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:38.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.369+0000 7fab16a00640 1 -- 172.21.15.82:0/151884146 >> v1:172.21.15.90:6789/0 conn(0x7fab18147060 legacy=0x7fab1806e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:38.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.369+0000 7fab16a00640 1 -- 172.21.15.82:0/151884146 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1806d490 con 0x7fab181469c0 2024-03-16T17:58:38.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.369+0000 7fab17400640 1 --2- 172.21.15.82:0/151884146 >> v2:172.21.15.90:3300/0 conn(0x7fab180cddc0 0x7fab180ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:58:38.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.370+0000 7fab16a00640 1 --2- 172.21.15.82:0/151884146 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 0x7fab18162360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab080097a0 tx=0x7fab0802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fe1d44286e66932 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:38.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.371+0000 7fab15600640 1 -- 172.21.15.82:0/151884146 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0802d040 con 0x7fab181469c0 2024-03-16T17:58:38.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.371+0000 7fab15600640 1 -- 172.21.15.82:0/151884146 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab08031810 con 0x7fab181469c0 2024-03-16T17:58:38.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.371+0000 7fab15600640 1 -- 172.21.15.82:0/151884146 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08031ba0 con 0x7fab181469c0 2024-03-16T17:58:38.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.372+0000 7fab1d800640 1 -- 172.21.15.82:0/151884146 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 msgr2=0x7fab18162360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:38.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.372+0000 7fab1d800640 1 --2- 172.21.15.82:0/151884146 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 0x7fab18162360 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab080097a0 tx=0x7fab0802b8b0 comp rx=0 tx=0).stop 2024-03-16T17:58:38.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.372+0000 7fab1d800640 1 -- 172.21.15.82:0/151884146 shutdown_connections 2024-03-16T17:58:38.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.372+0000 7fab1d800640 1 --2- 172.21.15.82:0/151884146 >> v2:172.21.15.90:3300/0 conn(0x7fab180cddc0 0x7fab180ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:38.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.372+0000 7fab1d800640 1 --2- 172.21.15.82:0/151884146 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 0x7fab18162360 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:38.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.372+0000 7fab1d800640 1 -- 172.21.15.82:0/151884146 >> 172.21.15.82:0/151884146 conn(0x7fab180b1640 msgr2=0x7fab180b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:38.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.372+0000 7fab1d800640 1 -- 172.21.15.82:0/151884146 shutdown_connections 2024-03-16T17:58:38.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.372+0000 7fab1d800640 1 -- 172.21.15.82:0/151884146 wait complete. 2024-03-16T17:58:38.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.374+0000 7fab1d800640 1 Processor -- start 2024-03-16T17:58:38.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.374+0000 7fab1d800640 1 -- start start 2024-03-16T17:58:38.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.375+0000 7fab1d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab180cddc0 0x7fab180b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:38.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.375+0000 7fab1d800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 0x7fab181424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:38.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.375+0000 7fab1d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab18147060 0x7fab1815a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:38.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.375+0000 7fab1d800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18143300 con 0x7fab18147060 2024-03-16T17:58:38.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.375+0000 7fab1d800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1815ad40 con 0x7fab180cddc0 2024-03-16T17:58:38.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.375+0000 7fab1d800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1815afe0 con 0x7fab181469c0 2024-03-16T17:58:38.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab17400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab18147060 0x7fab1815a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:38.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab17400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab18147060 0x7fab1815a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37740/0 (socket says 172.21.15.82:37740) 2024-03-16T17:58:38.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab17400640 1 -- 172.21.15.82:0/685137792 learned_addr learned my addr 172.21.15.82:0/685137792 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab16a00640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab180cddc0 0x7fab180b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab16000640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 0x7fab181424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab17400640 1 -- 172.21.15.82:0/685137792 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 msgr2=0x7fab181424a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab17400640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 0x7fab181424a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab17400640 1 -- 172.21.15.82:0/685137792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab180cddc0 msgr2=0x7fab180b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab17400640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab180cddc0 0x7fab180b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab17400640 1 -- 172.21.15.82:0/685137792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab10009a70 con 0x7fab18147060 2024-03-16T17:58:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.376+0000 7fab16a00640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab180cddc0 0x7fab180b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:38.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.377+0000 7fab17400640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab18147060 0x7fab1815a800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab1000efc0 tx=0x7fab10009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:38.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.377+0000 7fab0ea00640 1 -- 172.21.15.82:0/685137792 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10009d70 con 0x7fab18147060 2024-03-16T17:58:38.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.377+0000 7fab0ea00640 1 -- 172.21.15.82:0/685137792 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1000f040 con 0x7fab18147060 2024-03-16T17:58:38.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.377+0000 7fab1d800640 1 -- 172.21.15.82:0/685137792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab08009400 con 0x7fab18147060 2024-03-16T17:58:38.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.377+0000 7fab0ea00640 1 -- 172.21.15.82:0/685137792 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10003990 con 0x7fab18147060 2024-03-16T17:58:38.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.377+0000 7fab1d800640 1 -- 172.21.15.82:0/685137792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1815b5a0 con 0x7fab18147060 2024-03-16T17:58:38.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.378+0000 7fab1d800640 1 -- 172.21.15.82:0/685137792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab181434b0 con 0x7fab18147060 2024-03-16T17:58:38.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.379+0000 7fab0ea00640 1 -- 172.21.15.82:0/685137792 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fab100144a0 con 0x7fab18147060 2024-03-16T17:58:38.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.380+0000 7fab0ea00640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faaf806b3b0 0x7faaf806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:38.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.381+0000 7fab0ea00640 1 -- 172.21.15.82:0/685137792 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fab1008aa30 con 0x7fab18147060 2024-03-16T17:58:38.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.381+0000 7fab16a00640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faaf806b3b0 0x7faaf806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:38.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.381+0000 7fab16a00640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faaf806b3b0 0x7faaf806d870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab08036000 tx=0x7fab0803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:38.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.385+0000 7fab0ea00640 1 -- 172.21.15.82:0/685137792 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab181434b0 con 0x7fab18147060 2024-03-16T17:58:38.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:38.622+0000 7fab1d800640 1 -- 172.21.15.82:0/685137792 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fab181430f0 con 0x7fab18147060 2024-03-16T17:58:39.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.145+0000 7fab0ea00640 1 -- 172.21.15.82:0/685137792 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fab10059030 con 0x7fab18147060 2024-03-16T17:58:39.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.149+0000 7fab1d800640 1 -- 172.21.15.82:0/685137792 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faaf806b3b0 msgr2=0x7faaf806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:39.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.149+0000 7fab1d800640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faaf806b3b0 0x7faaf806d870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fab08036000 tx=0x7fab0803c040 comp rx=0 tx=0).stop 2024-03-16T17:58:39.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.149+0000 7fab1d800640 1 -- 172.21.15.82:0/685137792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab18147060 msgr2=0x7fab1815a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:39.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.149+0000 7fab1d800640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab18147060 0x7fab1815a800 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab1000efc0 tx=0x7fab10009f90 comp rx=0 tx=0).stop 2024-03-16T17:58:39.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.149+0000 7fab1d800640 1 -- 172.21.15.82:0/685137792 shutdown_connections 2024-03-16T17:58:39.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.149+0000 7fab1d800640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faaf806b3b0 0x7faaf806d870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:39.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.149+0000 7fab1d800640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab18147060 0x7fab1815a800 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:39.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.149+0000 7fab1d800640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fab181469c0 0x7fab181424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:39.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.149+0000 7fab1d800640 1 --2- 172.21.15.82:0/685137792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab180cddc0 0x7fab180b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:39.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.149+0000 7fab1d800640 1 -- 172.21.15.82:0/685137792 >> 172.21.15.82:0/685137792 conn(0x7fab180b1640 msgr2=0x7fab180b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:39.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.150+0000 7fab1d800640 1 -- 172.21.15.82:0/685137792 shutdown_connections 2024-03-16T17:58:39.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.150+0000 7fab1d800640 1 -- 172.21.15.82:0/685137792 wait complete. 2024-03-16T17:58:39.174 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-16T17:58:39.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.382+0000 7f8af5c00640 1 Processor -- start 2024-03-16T17:58:39.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.383+0000 7f8af5c00640 1 -- start start 2024-03-16T17:58:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.384+0000 7f8af5c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af00b31d0 0x7f8af0166c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.384+0000 7f8af5c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af00b0860 con 0x7f8af015f6a0 2024-03-16T17:58:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.384+0000 7f8af5c00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8af00b1140 con 0x7f8af00b1340 2024-03-16T17:58:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.384+0000 7f8af5c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00cc4f0 con 0x7f8af00b31d0 2024-03-16T17:58:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.384+0000 7f8aeea00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8af015f6a0 0x7f8af00b0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35726/0 (socket says 172.21.15.82:35726) 2024-03-16T17:58:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.384+0000 7f8aeea00640 1 -- 172.21.15.82:0/301711340 learned_addr learned my addr 172.21.15.82:0/301711340 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.385+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356517680 0 0) 0x7f8af00b0860 con 0x7f8af015f6a0 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.385+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad4003520 con 0x7f8af015f6a0 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.385+0000 7f8aef400640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af00b31d0 0x7f8af0166c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.385+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 516725264 0 0) 0x7f8ad4003520 con 0x7f8af015f6a0 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.385+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af00b31d0 msgr2=0x7f8af0166c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.385+0000 7f8aed600640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af00b31d0 0x7f8af0166c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.385+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 >> v1:172.21.15.90:6789/0 conn(0x7f8af00b1340 legacy=0x7f8af00b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.385+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af00afc50 con 0x7f8af015f6a0 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.385+0000 7f8aef400640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af00b31d0 0x7f8af0166c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.385+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f8ae00019f0 con 0x7f8af015f6a0 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 >> v1:172.21.15.82:6789/0 conn(0x7f8af015f6a0 legacy=0x7f8af00b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aed600640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af00b31d0 0x7f8ad40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:39.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aed600640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 0x7f8ad40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aed600640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ad4004d00 0x7f8ad4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af00cc4f0 con 0x7f8ad4003fe0 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad400bbd0 con 0x7f8ad4004d00 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad400bd90 con 0x7f8af00b31d0 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aee000640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 0x7f8ad40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aee000640 1 -- 172.21.15.82:0/301711340 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af00b31d0 msgr2=0x7f8ad40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aee000640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af00b31d0 0x7f8ad40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aee000640 1 -- 172.21.15.82:0/301711340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ad4004d00 msgr2=0x7f8ad4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aee000640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ad4004d00 0x7f8ad4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.386+0000 7f8aee000640 1 -- 172.21.15.82:0/301711340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4003520 con 0x7f8ad4003fe0 2024-03-16T17:58:39.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.387+0000 7f8aee000640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 0x7f8ad40047c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8001720 tx=0x7f8ad40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caf901b817f0fb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.388+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80052d0 con 0x7f8ad4003fe0 2024-03-16T17:58:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.388+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad802e050 con 0x7f8ad4003fe0 2024-03-16T17:58:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.388+0000 7f8aed600640 1 -- 172.21.15.82:0/301711340 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8003830 con 0x7f8ad4003fe0 2024-03-16T17:58:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.388+0000 7f8af5c00640 1 -- 172.21.15.82:0/301711340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 msgr2=0x7f8ad40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.388+0000 7f8af5c00640 1 --2- 172.21.15.82:0/301711340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 0x7f8ad40047c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8001720 tx=0x7f8ad40057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.388+0000 7f8aef400640 1 -- 172.21.15.82:0/301711340 reap_dead start 2024-03-16T17:58:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.388+0000 7f8af5c00640 1 -- 172.21.15.82:0/301711340 shutdown_connections 2024-03-16T17:58:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.388+0000 7f8af5c00640 1 -- 172.21.15.82:0/301711340 >> 172.21.15.82:0/301711340 conn(0x7f8af00d1b50 msgr2=0x7f8af00d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.388+0000 7f8af5c00640 1 -- 172.21.15.82:0/301711340 shutdown_connections 2024-03-16T17:58:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.388+0000 7f8af5c00640 1 -- 172.21.15.82:0/301711340 wait complete. 2024-03-16T17:58:39.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.390+0000 7f8af5c00640 1 Processor -- start 2024-03-16T17:58:39.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.390+0000 7f8af5c00640 1 -- start start 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.391+0000 7f8af5c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 0x7f8af0156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.391+0000 7f8af5c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8af0158280 0x7f8af014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.391+0000 7f8af5c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af015a840 0x7f8af015acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.391+0000 7f8af5c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af0158700 con 0x7f8ad4003fe0 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.391+0000 7f8af5c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8af015b200 con 0x7f8af0158280 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.391+0000 7f8af5c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8af006bf80 con 0x7f8af015a840 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.392+0000 7f8aeea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 0x7f8af0156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.392+0000 7f8aeea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 0x7f8af0156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37756/0 (socket says 172.21.15.82:37756) 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.392+0000 7f8aeea00640 1 -- 172.21.15.82:0/3253416033 learned_addr learned my addr 172.21.15.82:0/3253416033 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.392+0000 7f8aeea00640 1 -- 172.21.15.82:0/3253416033 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af015a840 msgr2=0x7f8af015acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.392+0000 7f8aeea00640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af015a840 0x7f8af015acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.392+0000 7f8aeea00640 1 -- 172.21.15.82:0/3253416033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8af0158280 msgr2=0x7f8af014af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:39.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.392+0000 7f8aeea00640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8af0158280 0x7f8af014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:39.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.392+0000 7f8aeea00640 1 -- 172.21.15.82:0/3253416033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af01522b0 con 0x7f8ad4003fe0 2024-03-16T17:58:39.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.392+0000 7f8aeea00640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 0x7f8af0156270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0001370 tx=0x7f8ae000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:39.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.392+0000 7f8ae6a00640 1 -- 172.21.15.82:0/3253416033 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000a7c0 con 0x7f8ad4003fe0 2024-03-16T17:58:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.393+0000 7f8ae6a00640 1 -- 172.21.15.82:0/3253416033 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae000a920 con 0x7f8ad4003fe0 2024-03-16T17:58:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.393+0000 7f8af5c00640 1 -- 172.21.15.82:0/3253416033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8af014b4c0 con 0x7f8ad4003fe0 2024-03-16T17:58:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.393+0000 7f8ae6a00640 1 -- 172.21.15.82:0/3253416033 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000acb0 con 0x7f8ad4003fe0 2024-03-16T17:58:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.393+0000 7f8af5c00640 1 -- 172.21.15.82:0/3253416033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8af0156be0 con 0x7f8ad4003fe0 2024-03-16T17:58:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.394+0000 7f8af5c00640 1 -- 172.21.15.82:0/3253416033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8af00d0760 con 0x7f8ad4003fe0 2024-03-16T17:58:39.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.395+0000 7f8ae6a00640 1 -- 172.21.15.82:0/3253416033 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8ae00070d0 con 0x7f8ad4003fe0 2024-03-16T17:58:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.395+0000 7f8ae6a00640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8acc06b3b0 0x7f8acc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.396+0000 7f8ae6a00640 1 -- 172.21.15.82:0/3253416033 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f8ae0089930 con 0x7f8ad4003fe0 2024-03-16T17:58:39.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.396+0000 7f8aee000640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8acc06b3b0 0x7f8acc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:39.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.396+0000 7f8aee000640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8acc06b3b0 0x7f8acc06d870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8033000 tx=0x7f8ad802f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:39.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.400+0000 7f8ae6a00640 1 -- 172.21.15.82:0/3253416033 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8ae0054400 con 0x7f8ad4003fe0 2024-03-16T17:58:39.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:39.597+0000 7f8af5c00640 1 -- 172.21.15.82:0/3253416033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8af0068640 con 0x7f8ad4003fe0 2024-03-16T17:58:40.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.325+0000 7f8ae6a00640 1 -- 172.21.15.82:0/3253416033 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8ae0018020 con 0x7f8ad4003fe0 2024-03-16T17:58:40.329 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-03-16T17:58:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.329+0000 7f8af5c00640 1 -- 172.21.15.82:0/3253416033 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8acc06b3b0 msgr2=0x7f8acc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.329+0000 7f8af5c00640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8acc06b3b0 0x7f8acc06d870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8033000 tx=0x7f8ad802f000 comp rx=0 tx=0).stop 2024-03-16T17:58:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.329+0000 7f8af5c00640 1 -- 172.21.15.82:0/3253416033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 msgr2=0x7f8af0156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.329+0000 7f8af5c00640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 0x7f8af0156270 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0001370 tx=0x7f8ae000cf00 comp rx=0 tx=0).stop 2024-03-16T17:58:40.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.330+0000 7f8af5c00640 1 -- 172.21.15.82:0/3253416033 shutdown_connections 2024-03-16T17:58:40.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.330+0000 7f8af5c00640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8acc06b3b0 0x7f8acc06d870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.330+0000 7f8af5c00640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8af015a840 0x7f8af015acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.330+0000 7f8af5c00640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8af0158280 0x7f8af014af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.330+0000 7f8af5c00640 1 --2- 172.21.15.82:0/3253416033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad4003fe0 0x7f8af0156270 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.330+0000 7f8af5c00640 1 -- 172.21.15.82:0/3253416033 >> 172.21.15.82:0/3253416033 conn(0x7f8af00d1b50 msgr2=0x7f8af00d1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:40.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.330+0000 7f8af5c00640 1 -- 172.21.15.82:0/3253416033 shutdown_connections 2024-03-16T17:58:40.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.331+0000 7f8af5c00640 1 -- 172.21.15.82:0/3253416033 wait complete. 2024-03-16T17:58:40.360 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-16T17:58:40.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.549+0000 7fcdfb200640 1 Processor -- start 2024-03-16T17:58:40.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.549+0000 7fcdfb200640 1 -- start start 2024-03-16T17:58:40.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.550+0000 7fcdfb200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fcdfc0b3330 0x7fcdfc06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:40.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.550+0000 7fcdfb200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcdfc071a60 0x7fcdfc071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:40.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.550+0000 7fcdfb200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3710 0x7fcdfc0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:40.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdfb200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdfc06c8b0 con 0x7fcdfc071a60 2024-03-16T17:58:40.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdfb200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdfc06cb90 con 0x7fcdfc0b3330 2024-03-16T17:58:40.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdfb200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc06ce70 con 0x7fcdfc0b3710 2024-03-16T17:58:40.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdf1400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcdfc071a60 0x7fcdfc071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:40.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdf1400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcdfc071a60 0x7fcdfc071ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37764/0 (socket says 172.21.15.82:37764) 2024-03-16T17:58:40.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdf1400640 1 -- 172.21.15.82:0/3818235498 learned_addr learned my addr 172.21.15.82:0/3818235498 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:40.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdfa800640 1 --2- 172.21.15.82:0/3818235498 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3710 0x7fcdfc0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:40.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdf1400640 1 -- 172.21.15.82:0/3818235498 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3710 msgr2=0x7fcdfc0b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:40.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdf1400640 1 --2- 172.21.15.82:0/3818235498 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3710 0x7fcdfc0b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdf1400640 1 -- 172.21.15.82:0/3818235498 >> v2:172.21.15.90:3300/0 conn(0x7fcdfc0b3330 msgr2=0x7fcdfc06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:40.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdf9e00640 1 --2- 172.21.15.82:0/3818235498 >> v2:172.21.15.90:3300/0 conn(0x7fcdfc0b3330 0x7fcdfc06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:40.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdf1400640 1 --2- 172.21.15.82:0/3818235498 >> v2:172.21.15.90:3300/0 conn(0x7fcdfc0b3330 0x7fcdfc06dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdf1400640 1 -- 172.21.15.82:0/3818235498 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc06d150 con 0x7fcdfc071a60 2024-03-16T17:58:40.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdfa800640 1 --2- 172.21.15.82:0/3818235498 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3710 0x7fcdfc0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:40.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.551+0000 7fcdf9e00640 1 --2- 172.21.15.82:0/3818235498 >> v2:172.21.15.90:3300/0 conn(0x7fcdfc0b3330 0x7fcdfc06dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:40.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.552+0000 7fcdf1400640 1 --2- 172.21.15.82:0/3818235498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc071a60 0x7fcdfc071ec0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc06be50 tx=0x7fcde402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21efe70a12938d00 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:40.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.553+0000 7fcdf9400640 1 -- 172.21.15.82:0/3818235498 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4031860 con 0x7fcdfc071a60 2024-03-16T17:58:40.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.553+0000 7fcdf9400640 1 -- 172.21.15.82:0/3818235498 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde40319c0 con 0x7fcdfc071a60 2024-03-16T17:58:40.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.553+0000 7fcdf9400640 1 -- 172.21.15.82:0/3818235498 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4031d50 con 0x7fcdfc071a60 2024-03-16T17:58:40.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.553+0000 7fcdfb200640 1 -- 172.21.15.82:0/3818235498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc071a60 msgr2=0x7fcdfc071ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:40.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.553+0000 7fcdfb200640 1 --2- 172.21.15.82:0/3818235498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc071a60 0x7fcdfc071ec0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc06be50 tx=0x7fcde402b900 comp rx=0 tx=0).stop 2024-03-16T17:58:40.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.553+0000 7fcdfb200640 1 -- 172.21.15.82:0/3818235498 shutdown_connections 2024-03-16T17:58:40.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.553+0000 7fcdfb200640 1 --2- 172.21.15.82:0/3818235498 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3710 0x7fcdfc0b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.553+0000 7fcdfb200640 1 --2- 172.21.15.82:0/3818235498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc071a60 0x7fcdfc071ec0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.553+0000 7fcdfb200640 1 --2- 172.21.15.82:0/3818235498 >> v2:172.21.15.90:3300/0 conn(0x7fcdfc0b3330 0x7fcdfc06dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.553+0000 7fcdfb200640 1 -- 172.21.15.82:0/3818235498 >> 172.21.15.82:0/3818235498 conn(0x7fcdfc06b050 msgr2=0x7fcdfc06b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.554+0000 7fcdfb200640 1 -- 172.21.15.82:0/3818235498 shutdown_connections 2024-03-16T17:58:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.554+0000 7fcdfb200640 1 -- 172.21.15.82:0/3818235498 wait complete. 2024-03-16T17:58:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.555+0000 7fcdfb200640 1 Processor -- start 2024-03-16T17:58:40.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.556+0000 7fcdfb200640 1 -- start start 2024-03-16T17:58:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.556+0000 7fcdfb200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdfc071a60 0x7fcdfc144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfb200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3330 0x7fcdfc145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfb200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc0b3710 0x7fcdfc164ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfb200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc163e90 con 0x7fcdfc0b3710 2024-03-16T17:58:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfb200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc163280 con 0x7fcdfc071a60 2024-03-16T17:58:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfb200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc163560 con 0x7fcdfc0b3330 2024-03-16T17:58:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfa800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc0b3710 0x7fcdfc164ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfa800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc0b3710 0x7fcdfc164ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37768/0 (socket says 172.21.15.82:37768) 2024-03-16T17:58:40.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfa800640 1 -- 172.21.15.82:0/1710256576 learned_addr learned my addr 172.21.15.82:0/1710256576 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdf9e00640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdfc071a60 0x7fcdfc144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdf1400640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3330 0x7fcdfc145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfa800640 1 -- 172.21.15.82:0/1710256576 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3330 msgr2=0x7fcdfc145060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfa800640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3330 0x7fcdfc145060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfa800640 1 -- 172.21.15.82:0/1710256576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdfc071a60 msgr2=0x7fcdfc144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfa800640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdfc071a60 0x7fcdfc144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.557+0000 7fcdfa800640 1 -- 172.21.15.82:0/1710256576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40094d0 con 0x7fcdfc0b3710 2024-03-16T17:58:40.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.558+0000 7fcdfa800640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc0b3710 0x7fcdfc164ba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400cef0 tx=0x7fcdf400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:40.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.558+0000 7fcdf2a00640 1 -- 172.21.15.82:0/1710256576 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf400d840 con 0x7fcdfc0b3710 2024-03-16T17:58:40.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.558+0000 7fcdf2a00640 1 -- 172.21.15.82:0/1710256576 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf400d9c0 con 0x7fcdfc0b3710 2024-03-16T17:58:40.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.558+0000 7fcdf2a00640 1 -- 172.21.15.82:0/1710256576 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf400dd70 con 0x7fcdfc0b3710 2024-03-16T17:58:40.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.558+0000 7fcdfb200640 1 -- 172.21.15.82:0/1710256576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc163840 con 0x7fcdfc0b3710 2024-03-16T17:58:40.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.558+0000 7fcdfb200640 1 -- 172.21.15.82:0/1710256576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc1faae0 con 0x7fcdfc0b3710 2024-03-16T17:58:40.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.559+0000 7fcdfb200640 1 -- 172.21.15.82:0/1710256576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc0ceb70 con 0x7fcdfc0b3710 2024-03-16T17:58:40.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.560+0000 7fcdf2a00640 1 -- 172.21.15.82:0/1710256576 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fcdf401e3b0 con 0x7fcdfc0b3710 2024-03-16T17:58:40.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.561+0000 7fcdf2a00640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcdd806b3b0 0x7fcdd806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:40.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.561+0000 7fcdf2a00640 1 -- 172.21.15.82:0/1710256576 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fcdf408a5b0 con 0x7fcdfc0b3710 2024-03-16T17:58:40.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.561+0000 7fcdf9e00640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcdd806b3b0 0x7fcdd806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:40.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.562+0000 7fcdf9e00640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcdd806b3b0 0x7fcdd806d870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00a950 tx=0x7fcdec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:40.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.565+0000 7fcdf2a00640 1 -- 172.21.15.82:0/1710256576 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcdf4055390 con 0x7fcdfc0b3710 2024-03-16T17:58:40.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:40.761+0000 7fcdfb200640 1 -- 172.21.15.82:0/1710256576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcdfc0b4090 con 0x7fcdfc0b3710 2024-03-16T17:58:41.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.329+0000 7fcdf2a00640 1 -- 172.21.15.82:0/1710256576 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcdf4058a50 con 0x7fcdfc0b3710 2024-03-16T17:58:41.333 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-03-16T17:58:41.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.334+0000 7fcdfb200640 1 -- 172.21.15.82:0/1710256576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcdd806b3b0 msgr2=0x7fcdd806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:41.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.334+0000 7fcdfb200640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcdd806b3b0 0x7fcdd806d870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00a950 tx=0x7fcdec008040 comp rx=0 tx=0).stop 2024-03-16T17:58:41.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.334+0000 7fcdfb200640 1 -- 172.21.15.82:0/1710256576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc0b3710 msgr2=0x7fcdfc164ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:41.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.334+0000 7fcdfb200640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc0b3710 0x7fcdfc164ba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400cef0 tx=0x7fcdf400a710 comp rx=0 tx=0).stop 2024-03-16T17:58:41.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.337+0000 7fcdfb200640 1 -- 172.21.15.82:0/1710256576 shutdown_connections 2024-03-16T17:58:41.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.337+0000 7fcdfb200640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcdd806b3b0 0x7fcdd806d870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:41.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.337+0000 7fcdfb200640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdfc0b3710 0x7fcdfc164ba0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:41.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.337+0000 7fcdfb200640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcdfc0b3330 0x7fcdfc145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:41.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.337+0000 7fcdfb200640 1 --2- 172.21.15.82:0/1710256576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcdfc071a60 0x7fcdfc144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:41.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.337+0000 7fcdfb200640 1 -- 172.21.15.82:0/1710256576 >> 172.21.15.82:0/1710256576 conn(0x7fcdfc06b050 msgr2=0x7fcdfc161d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:41.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.337+0000 7fcdfb200640 1 -- 172.21.15.82:0/1710256576 shutdown_connections 2024-03-16T17:58:41.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.339+0000 7fcdfb200640 1 -- 172.21.15.82:0/1710256576 wait complete. 2024-03-16T17:58:41.368 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-16T17:58:41.369 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-16T17:58:41.369 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-16T17:58:41.369 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T17:58:41.369 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-16T17:58:41.369 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-16T17:58:41.369 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-16T17:58:41.369 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T17:58:41.370 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-16T17:58:41.370 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-16T17:58:41.370 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-16T17:58:41.370 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T17:58:41.370 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-16T17:58:41.370 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-16T17:58:41.370 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-16T17:58:41.371 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T17:58:41.388 INFO:tasks.ceph.mds.c:Stopped 2024-03-16T17:58:41.388 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-16T17:58:41.389 INFO:tasks.ceph.mds.a:Stopped 2024-03-16T17:58:41.389 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-16T17:58:41.399 INFO:tasks.ceph.mds.d:Stopped 2024-03-16T17:58:41.399 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-16T17:58:41.400 INFO:tasks.ceph.mds.b:Stopped 2024-03-16T17:58:41.400 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-16T17:58:41.436 INFO:tasks.ceph.mds.c:Started 2024-03-16T17:58:41.436 DEBUG:teuthology.parallel:result is None 2024-03-16T17:58:41.442 INFO:tasks.ceph.mds.a:Started 2024-03-16T17:58:41.442 DEBUG:teuthology.parallel:result is None 2024-03-16T17:58:41.447 INFO:tasks.ceph.mds.d:Started 2024-03-16T17:58:41.447 DEBUG:teuthology.parallel:result is None 2024-03-16T17:58:41.453 INFO:tasks.ceph.mds.b:Started 2024-03-16T17:58:41.453 DEBUG:teuthology.parallel:result is None 2024-03-16T17:58:41.453 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-16T17:58:41.585 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-03-16T17:58:41.587 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-03-16T17:58:41.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-03-16T17:58:41.612 INFO:tasks.ceph.mds.d.smithi090.stdout:starting mds.d at 2024-03-16T17:58:41.612 INFO:tasks.ceph.mds.b.smithi090.stdout:starting mds.b at 2024-03-16T17:58:41.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.624+0000 7f8d5a400640 1 Processor -- start 2024-03-16T17:58:41.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.624+0000 7f8d5a400640 1 -- start start 2024-03-16T17:58:41.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.625+0000 7f8d5a400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d5415df00 0x7f8d540cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:41.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d5a400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8d540b1000 0x7f8d540cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:41.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d5a400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5415e2e0 con 0x7f8d540cd890 2024-03-16T17:58:41.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d5a400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540ccec0 con 0x7f8d540b1000 2024-03-16T17:58:41.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d5a400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540ced50 con 0x7f8d5415df00 2024-03-16T17:58:41.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d59a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8d540cd890 0x7f8d540cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35756/0 (socket says 172.21.15.82:35756) 2024-03-16T17:58:41.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d59a00640 1 -- 172.21.15.82:0/4251209511 learned_addr learned my addr 172.21.15.82:0/4251209511 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:41.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d59000640 1 --2- 172.21.15.82:0/4251209511 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d5415df00 0x7f8d540cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:41.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d4be00640 1 --2- 172.21.15.82:0/4251209511 >> v2:172.21.15.90:3300/0 conn(0x7f8d540b1000 0x7f8d540cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:41.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d53e00640 1 -- 172.21.15.82:0/4251209511 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690568107 0 0) 0x7f8d4c001490 con 0x7f8d540cd890 2024-03-16T17:58:41.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d59000640 1 -- 172.21.15.82:0/4251209511 >> v2:172.21.15.90:3300/0 conn(0x7f8d540b1000 msgr2=0x7f8d540cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:41.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d59000640 1 --2- 172.21.15.82:0/4251209511 >> v2:172.21.15.90:3300/0 conn(0x7f8d540b1000 0x7f8d540cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:41.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d59000640 1 -- 172.21.15.82:0/4251209511 >> v1:172.21.15.82:6789/0 conn(0x7f8d540cd890 legacy=0x7f8d540cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:41.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.626+0000 7f8d59000640 1 -- 172.21.15.82:0/4251209511 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540cefd0 con 0x7f8d5415df00 2024-03-16T17:58:41.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.627+0000 7f8d59000640 1 --2- 172.21.15.82:0/4251209511 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d5415df00 0x7f8d540cf5e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d440097a0 tx=0x7f8d4402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbe632ed1ca0649 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d53e00640 1 -- 172.21.15.82:0/4251209511 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44031830 con 0x7f8d5415df00 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d53e00640 1 -- 172.21.15.82:0/4251209511 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d44031990 con 0x7f8d5415df00 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d53e00640 1 -- 172.21.15.82:0/4251209511 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44031d20 con 0x7f8d5415df00 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d5a400640 1 -- 172.21.15.82:0/4251209511 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d5415df00 msgr2=0x7f8d540cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d5a400640 1 --2- 172.21.15.82:0/4251209511 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d5415df00 0x7f8d540cf5e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d440097a0 tx=0x7f8d4402b8d0 comp rx=0 tx=0).stop 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d5a400640 1 -- 172.21.15.82:0/4251209511 shutdown_connections 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d5a400640 1 --2- 172.21.15.82:0/4251209511 >> v2:172.21.15.90:3300/0 conn(0x7f8d540b1000 0x7f8d540cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d5a400640 1 --2- 172.21.15.82:0/4251209511 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d5415df00 0x7f8d540cf5e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d5a400640 1 -- 172.21.15.82:0/4251209511 >> 172.21.15.82:0/4251209511 conn(0x7f8d540b3600 msgr2=0x7f8d540b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d5a400640 1 -- 172.21.15.82:0/4251209511 shutdown_connections 2024-03-16T17:58:41.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.628+0000 7f8d5a400640 1 -- 172.21.15.82:0/4251209511 wait complete. 2024-03-16T17:58:41.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.630+0000 7f8d5a400640 1 Processor -- start 2024-03-16T17:58:41.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.630+0000 7f8d5a400640 1 -- start start 2024-03-16T17:58:41.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d5a400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d540b1000 0x7f8d54159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:41.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d5a400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d540cd890 0x7f8d541f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:41.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d5a400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d5415df00 0x7f8d541f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:41.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d5a400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540b5720 con 0x7f8d540b1000 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d5a400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d541f7f40 con 0x7f8d5415df00 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d5a400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540b5030 con 0x7f8d540cd890 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d59000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d540b1000 0x7f8d54159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d59000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d540b1000 0x7f8d54159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37816/0 (socket says 172.21.15.82:37816) 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d59000640 1 -- 172.21.15.82:0/1231948926 learned_addr learned my addr 172.21.15.82:0/1231948926 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d4be00640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d540cd890 0x7f8d541f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d59a00640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d5415df00 0x7f8d541f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d59000640 1 -- 172.21.15.82:0/1231948926 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d540cd890 msgr2=0x7f8d541f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d59000640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d540cd890 0x7f8d541f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d59000640 1 -- 172.21.15.82:0/1231948926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d5415df00 msgr2=0x7f8d541f7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d59000640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d5415df00 0x7f8d541f7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.631+0000 7f8d59000640 1 -- 172.21.15.82:0/1231948926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d44009400 con 0x7f8d540b1000 2024-03-16T17:58:41.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.632+0000 7f8d59000640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d540b1000 0x7f8d54159e80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8d44034040 tx=0x7f8d440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:41.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.632+0000 7f8d51600640 1 -- 172.21.15.82:0/1231948926 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4403a8f0 con 0x7f8d540b1000 2024-03-16T17:58:41.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.632+0000 7f8d51600640 1 -- 172.21.15.82:0/1231948926 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4403aa70 con 0x7f8d540b1000 2024-03-16T17:58:41.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.632+0000 7f8d51600640 1 -- 172.21.15.82:0/1231948926 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4403ae20 con 0x7f8d540b1000 2024-03-16T17:58:41.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.632+0000 7f8d5a400640 1 -- 172.21.15.82:0/1231948926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d541f7c60 con 0x7f8d540b1000 2024-03-16T17:58:41.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.632+0000 7f8d5a400640 1 -- 172.21.15.82:0/1231948926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d541f3ef0 con 0x7f8d540b1000 2024-03-16T17:58:41.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.633+0000 7f8d5a400640 1 -- 172.21.15.82:0/1231948926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d540689b0 con 0x7f8d540b1000 2024-03-16T17:58:41.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.634+0000 7f8d51600640 1 -- 172.21.15.82:0/1231948926 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8d4404d440 con 0x7f8d540b1000 2024-03-16T17:58:41.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.635+0000 7f8d51600640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8d3006b3b0 0x7f8d3006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:41.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.635+0000 7f8d51600640 1 -- 172.21.15.82:0/1231948926 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8d440ab870 con 0x7f8d540b1000 2024-03-16T17:58:41.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.635+0000 7f8d4be00640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8d3006b3b0 0x7f8d3006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:41.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.636+0000 7f8d4be00640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8d3006b3b0 0x7f8d3006d870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d540c84b0 tx=0x7f8d3c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:41.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.639+0000 7f8d51600640 1 -- 172.21.15.82:0/1231948926 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8d44076880 con 0x7f8d540b1000 2024-03-16T17:58:41.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:41.847+0000 7f8d5a400640 1 -- 172.21.15.82:0/1231948926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8d5404f250 con 0x7f8d540b1000 2024-03-16T17:58:42.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.367+0000 7f8d51600640 1 -- 172.21.15.82:0/1231948926 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8d44035020 con 0x7f8d540b1000 2024-03-16T17:58:42.371 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-03-16T17:58:42.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.370+0000 7f8d5a400640 1 -- 172.21.15.82:0/1231948926 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8d3006b3b0 msgr2=0x7f8d3006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:42.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.370+0000 7f8d5a400640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8d3006b3b0 0x7f8d3006d870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d540c84b0 tx=0x7f8d3c009190 comp rx=0 tx=0).stop 2024-03-16T17:58:42.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.370+0000 7f8d5a400640 1 -- 172.21.15.82:0/1231948926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d540b1000 msgr2=0x7f8d54159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:42.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.370+0000 7f8d5a400640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d540b1000 0x7f8d54159e80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8d44034040 tx=0x7f8d440046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.370+0000 7f8d5a400640 1 -- 172.21.15.82:0/1231948926 shutdown_connections 2024-03-16T17:58:42.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.370+0000 7f8d5a400640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8d3006b3b0 0x7f8d3006d870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.370+0000 7f8d5a400640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d5415df00 0x7f8d541f7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.370+0000 7f8d5a400640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8d540cd890 0x7f8d541f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.370+0000 7f8d5a400640 1 --2- 172.21.15.82:0/1231948926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d540b1000 0x7f8d54159e80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.370+0000 7f8d5a400640 1 -- 172.21.15.82:0/1231948926 >> 172.21.15.82:0/1231948926 conn(0x7f8d540b3600 msgr2=0x7f8d540b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:42.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.371+0000 7f8d5a400640 1 -- 172.21.15.82:0/1231948926 shutdown_connections 2024-03-16T17:58:42.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.371+0000 7f8d5a400640 1 -- 172.21.15.82:0/1231948926 wait complete. 2024-03-16T17:58:42.397 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-16T17:58:42.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.593+0000 7feb4d800640 1 Processor -- start 2024-03-16T17:58:42.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.593+0000 7feb4d800640 1 -- start start 2024-03-16T17:58:42.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.594+0000 7feb4d800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb481454d0 0x7feb480b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:42.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.594+0000 7feb4d800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7feb480d0040 0x7feb48070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:42.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.594+0000 7feb4d800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb480b4b90 con 0x7feb480cdf80 2024-03-16T17:58:42.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.594+0000 7feb4d800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb480cf500 con 0x7feb480d0040 2024-03-16T17:58:42.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.594+0000 7feb4d800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb481458b0 con 0x7feb481454d0 2024-03-16T17:58:42.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb3e000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feb480cdf80 0x7feb480cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35770/0 (socket says 172.21.15.82:35770) 2024-03-16T17:58:42.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb3e000640 1 -- 172.21.15.82:0/964284324 learned_addr learned my addr 172.21.15.82:0/964284324 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:42.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb46a00640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb481454d0 0x7feb480b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:42.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb47400640 1 --2- 172.21.15.82:0/964284324 >> v2:172.21.15.90:3300/0 conn(0x7feb480d0040 0x7feb48070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:42.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936658974 0 0) 0x7feb480b4b90 con 0x7feb480cdf80 2024-03-16T17:58:42.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb2c003520 con 0x7feb480cdf80 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4268846923 0 0) 0x7feb2c003520 con 0x7feb480cdf80 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb481454d0 msgr2=0x7feb480b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb46000640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb481454d0 0x7feb480b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 >> v2:172.21.15.90:3300/0 conn(0x7feb480d0040 msgr2=0x7feb48070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.595+0000 7feb46000640 1 --2- 172.21.15.82:0/964284324 >> v2:172.21.15.90:3300/0 conn(0x7feb480d0040 0x7feb48070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb480ce460 con 0x7feb480cdf80 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7feb300019f0 con 0x7feb480cdf80 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 >> v1:172.21.15.82:6789/0 conn(0x7feb480cdf80 legacy=0x7feb480cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46000640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 0x7feb2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46000640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb2c0047c0 0x7feb2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:42.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46000640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb481454d0 0x7feb2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb480cf500 con 0x7feb2c003970 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb481458b0 con 0x7feb2c0047c0 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c00b980 con 0x7feb481454d0 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46a00640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 0x7feb2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb3e000640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb2c0047c0 0x7feb2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb47400640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb481454d0 0x7feb2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46a00640 1 -- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb481454d0 msgr2=0x7feb2c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46a00640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb481454d0 0x7feb2c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46a00640 1 -- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb2c0047c0 msgr2=0x7feb2c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46a00640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb2c0047c0 0x7feb2c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.596+0000 7feb46a00640 1 -- 172.21.15.82:0/964284324 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c003520 con 0x7feb2c003970 2024-03-16T17:58:42.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.597+0000 7feb46a00640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 0x7feb2c003d90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feb38009ac0 tx=0x7feb2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ec4b940cfbe10d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:42.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.598+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3802c6b0 con 0x7feb2c003970 2024-03-16T17:58:42.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.598+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3802c810 con 0x7feb2c003970 2024-03-16T17:58:42.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.598+0000 7feb46000640 1 -- 172.21.15.82:0/964284324 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3802cba0 con 0x7feb2c003970 2024-03-16T17:58:42.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.598+0000 7feb4d800640 1 -- 172.21.15.82:0/964284324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 msgr2=0x7feb2c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:42.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.598+0000 7feb4d800640 1 --2- 172.21.15.82:0/964284324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 0x7feb2c003d90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7feb38009ac0 tx=0x7feb2c0057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.598+0000 7feb47400640 1 -- 172.21.15.82:0/964284324 reap_dead start 2024-03-16T17:58:42.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.598+0000 7feb4d800640 1 -- 172.21.15.82:0/964284324 shutdown_connections 2024-03-16T17:58:42.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.598+0000 7feb4d800640 1 -- 172.21.15.82:0/964284324 >> 172.21.15.82:0/964284324 conn(0x7feb4806ae70 msgr2=0x7feb4806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:42.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.599+0000 7feb4d800640 1 -- 172.21.15.82:0/964284324 shutdown_connections 2024-03-16T17:58:42.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.599+0000 7feb4d800640 1 -- 172.21.15.82:0/964284324 wait complete. 2024-03-16T17:58:42.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.600+0000 7feb4d800640 1 Processor -- start 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.600+0000 7feb4d800640 1 -- start start 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.601+0000 7feb4d800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 0x7feb481f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.601+0000 7feb4d800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb48140800 0x7feb480aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.601+0000 7feb4d800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb480aa600 0x7feb480a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.601+0000 7feb4d800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb481f5320 con 0x7feb2c003970 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.601+0000 7feb4d800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb481f49c0 con 0x7feb480aa600 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.601+0000 7feb4d800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb480aaaa0 con 0x7feb48140800 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb46a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 0x7feb481f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb46a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 0x7feb481f0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37830/0 (socket says 172.21.15.82:37830) 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb46a00640 1 -- 172.21.15.82:0/106662803 learned_addr learned my addr 172.21.15.82:0/106662803 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:42.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb3e000640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb48140800 0x7feb480aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb46a00640 1 -- 172.21.15.82:0/106662803 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb48140800 msgr2=0x7feb480aa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb47400640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb480aa600 0x7feb480a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb46a00640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb48140800 0x7feb480aa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb46a00640 1 -- 172.21.15.82:0/106662803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb480aa600 msgr2=0x7feb480a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb46a00640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb480aa600 0x7feb480a9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb46a00640 1 -- 172.21.15.82:0/106662803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb481f4c60 con 0x7feb2c003970 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.602+0000 7feb46a00640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 0x7feb481f0cf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb38000c00 tx=0x7feb38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.603+0000 7feb3f400640 1 -- 172.21.15.82:0/106662803 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38003810 con 0x7feb2c003970 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.603+0000 7feb3f400640 1 -- 172.21.15.82:0/106662803 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb38003970 con 0x7feb2c003970 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.603+0000 7feb3f400640 1 -- 172.21.15.82:0/106662803 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38003ce0 con 0x7feb2c003970 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.603+0000 7feb4d800640 1 -- 172.21.15.82:0/106662803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb481f4f40 con 0x7feb2c003970 2024-03-16T17:58:42.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.603+0000 7feb4d800640 1 -- 172.21.15.82:0/106662803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb481f5740 con 0x7feb2c003970 2024-03-16T17:58:42.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.604+0000 7feb4d800640 1 -- 172.21.15.82:0/106662803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb48067f30 con 0x7feb2c003970 2024-03-16T17:58:42.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.605+0000 7feb3f400640 1 -- 172.21.15.82:0/106662803 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7feb38034440 con 0x7feb2c003970 2024-03-16T17:58:42.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.606+0000 7feb3f400640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7feb2406b3b0 0x7feb2406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:42.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.606+0000 7feb3f400640 1 -- 172.21.15.82:0/106662803 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feb380abc90 con 0x7feb2c003970 2024-03-16T17:58:42.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.606+0000 7feb3e000640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7feb2406b3b0 0x7feb2406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:42.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.607+0000 7feb3e000640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7feb2406b3b0 0x7feb2406d870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb4815a8c0 tx=0x7feb30002000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:42.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.610+0000 7feb3f400640 1 -- 172.21.15.82:0/106662803 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feb38076d70 con 0x7feb2c003970 2024-03-16T17:58:42.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.874+0000 7feb4d800640 1 -- 172.21.15.82:0/106662803 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7feb480b4640 con 0x7feb2c003970 2024-03-16T17:58:42.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.875+0000 7feb3f400640 1 -- 172.21.15.82:0/106662803 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7feb3804c090 con 0x7feb2c003970 2024-03-16T17:58:42.879 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:42.879 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T17:58:42.879 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-03-16T17:58:42.879 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBX3fVlBK4TDxAAgYBMjjtLq7Ts2ELUHOfrTw==", 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBX3fVlKwweMBAAlSH59V+WRQ7pPFGRQihe0g==", 2024-03-16T17:58:42.880 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBX3fVlZqfLHxAAr8uX4ldhnbNd3HwZwM3Ljw==", 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-16T17:58:42.881 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBY3fVl1XMTBRAAnwrP8FPBLSLM+/rR39Vu7Q==", 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.882 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBc3fVlYSYRMBAA1Tc0pI8roY4iOYXJLRb3yQ==", 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBg3fVlQYBLDRAAx8fhhK/pE3YSmSzTk0TC6A==", 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.883 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBj3fVlRNzLJxAAF2Kr2rtYnU1s3NRxt/w6dg==", 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.884 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBn3fVln2BkBBAAkAPCd6Z4ShuJUtZx62MmHA==", 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBu3fVlGZ38GxAAtcYQv0Efbj5ay4vsakaAjQ==", 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-16T17:58:42.885 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBx3fVlyRiYNBAArUz3DTmHVPqCm+BXw0W5vg==", 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.886 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQB13fVl7NTiAhAAJFQwOdPJqIKyeRencS6JPw==", 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQB43fVl87VCDBAAg8jmXVeiQskKjhT2pSHNuA==", 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-03-16T17:58:42.887 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBY3fVlVabyEhAAXRXAiI2UCfBSDg0N+ezAyw==", 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.888 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBY3fVlV/uMKBAAbU4pVWf+Fs33591f43297Q==", 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBY3fVlkjG3HRAAn0f/SF1SsTzAJ0wRpePYsA==", 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.889 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBY3fVlMOHVMhAAuOUJ6OGv536nnAmMLACc8w==", 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-03-16T17:58:42.890 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBW3fVlHzMbAhAAPrAs7qHZeUgs4cGOFpD7ow==", 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.891 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-mds", 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQCB3fVluPJYBxAAWZigjwrD94l0ugXztnmtaQ==", 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-mds" 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-mgr", 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQCB3fVlCg5ZBxAAPrQemQqulQ5TLQnpYiBmNQ==", 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-osd", 2024-03-16T17:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQCB3fVloiVZBxAAZYMGbXkS4rcmzWtp7ZepBQ==", 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-osd" 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-rbd", 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQCB3fVltDxZBxAAKOvcKX6saiATQP5nMcSbag==", 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-16T17:58:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQCB3fVlklRZBxAAMSu+XDTId7zLzmJSZOFZkA==", 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-rgw", 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQCB3fVlA21ZBxAADRdlL6uZLjLtLoEeHoUIig==", 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-03-16T17:58:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBX3fVl1ZlxAxAAHllQAsGYtDLSy1iO0jKGug==", 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQBW3fVlIaX+MhAAN42Ezb1GwDDzSV3Nx1ZcbQ==", 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-03-16T17:58:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.896 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:42.896 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-03-16T17:58:42.896 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 -- 172.21.15.82:0/106662803 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7feb2406b3b0 msgr2=0x7feb2406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7feb2406b3b0 0x7feb2406d870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feb4815a8c0 tx=0x7feb30002000 comp rx=0 tx=0).stop 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 -- 172.21.15.82:0/106662803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 msgr2=0x7feb481f0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 0x7feb481f0cf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb38000c00 tx=0x7feb38004730 comp rx=0 tx=0).stop 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 -- 172.21.15.82:0/106662803 shutdown_connections 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7feb2406b3b0 0x7feb2406d870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb480aa600 0x7feb480a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7feb48140800 0x7feb480aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 --2- 172.21.15.82:0/106662803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb2c003970 0x7feb481f0cf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 -- 172.21.15.82:0/106662803 >> 172.21.15.82:0/106662803 conn(0x7feb4806ae70 msgr2=0x7feb480cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:42.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 -- 172.21.15.82:0/106662803 shutdown_connections 2024-03-16T17:58:42.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:42.878+0000 7feb4d800640 1 -- 172.21.15.82:0/106662803 wait complete. 2024-03-16T17:58:42.899 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-16T17:58:43.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.096+0000 7f323f400640 1 Processor -- start 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.096+0000 7f323f400640 1 -- start start 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323f400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f323815df90 0x7f32380cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323f400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f323815e370 0x7f32380d0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323f400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32380d0e20 con 0x7f32380d0980 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323f400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32380cf180 con 0x7f323815df90 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323f400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32380ceac0 con 0x7f323815e370 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323ca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f323815df90 0x7f32380cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323ca00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f323815df90 0x7f32380cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37842/0 (socket says 172.21.15.82:37842) 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323ca00640 1 -- 172.21.15.82:0/2700518948 learned_addr learned my addr 172.21.15.82:0/2700518948 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323d400640 1 --1- 172.21.15.82:0/2700518948 >> v1:172.21.15.82:6789/0 conn(0x7f32380d0980 0x7f32380d23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35774/0 (socket says 172.21.15.82:35774) 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323ca00640 1 -- 172.21.15.82:0/2700518948 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f323815e370 msgr2=0x7f32380d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:43.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.098+0000 7f323ca00640 1 --2- 172.21.15.82:0/2700518948 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f323815e370 0x7f32380d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.099+0000 7f323ca00640 1 -- 172.21.15.82:0/2700518948 >> v1:172.21.15.82:6789/0 conn(0x7f32380d0980 legacy=0x7f32380d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.099+0000 7f323ca00640 1 -- 172.21.15.82:0/2700518948 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32380ceda0 con 0x7f323815df90 2024-03-16T17:58:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.100+0000 7f323ca00640 1 --2- 172.21.15.82:0/2700518948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815df90 0x7f32380cf5a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3228009870 tx=0x7f322802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00bbc0d44a1b41d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.100+0000 7f3237400640 1 -- 172.21.15.82:0/2700518948 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32280317c0 con 0x7f323815df90 2024-03-16T17:58:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.100+0000 7f3237400640 1 -- 172.21.15.82:0/2700518948 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3228031920 con 0x7f323815df90 2024-03-16T17:58:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.100+0000 7f3237400640 1 -- 172.21.15.82:0/2700518948 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228031cb0 con 0x7f323815df90 2024-03-16T17:58:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.100+0000 7f323f400640 1 -- 172.21.15.82:0/2700518948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815df90 msgr2=0x7f32380cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.100+0000 7f323f400640 1 --2- 172.21.15.82:0/2700518948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815df90 0x7f32380cf5a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3228009870 tx=0x7f322802b860 comp rx=0 tx=0).stop 2024-03-16T17:58:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.100+0000 7f323f400640 1 -- 172.21.15.82:0/2700518948 shutdown_connections 2024-03-16T17:58:43.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.100+0000 7f323f400640 1 --2- 172.21.15.82:0/2700518948 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f323815e370 0x7f32380d0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.100+0000 7f323f400640 1 --2- 172.21.15.82:0/2700518948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815df90 0x7f32380cf5a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.100+0000 7f323f400640 1 -- 172.21.15.82:0/2700518948 >> 172.21.15.82:0/2700518948 conn(0x7f323809d0e0 msgr2=0x7f32380d3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.101+0000 7f323f400640 1 -- 172.21.15.82:0/2700518948 shutdown_connections 2024-03-16T17:58:43.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.101+0000 7f323f400640 1 -- 172.21.15.82:0/2700518948 wait complete. 2024-03-16T17:58:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.102+0000 7f323f400640 1 Processor -- start 2024-03-16T17:58:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.102+0000 7f323f400640 1 -- start start 2024-03-16T17:58:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.103+0000 7f323f400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f32380d0980 0x7f32381569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.103+0000 7f323f400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f323815df90 0x7f3238155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.103+0000 7f323f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815e370 0x7f3238155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.103+0000 7f323f400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3238156390 con 0x7f323815e370 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.103+0000 7f323f400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32380ca120 con 0x7f323815df90 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.103+0000 7f323f400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3238156f20 con 0x7f32380d0980 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.103+0000 7f323d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815e370 0x7f3238155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.103+0000 7f323d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815e370 0x7f3238155e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37844/0 (socket says 172.21.15.82:37844) 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.103+0000 7f323d400640 1 -- 172.21.15.82:0/1519660427 learned_addr learned my addr 172.21.15.82:0/1519660427 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f323ca00640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f32380d0980 0x7f32381569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f3237e00640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f323815df90 0x7f3238155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f323d400640 1 -- 172.21.15.82:0/1519660427 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f32380d0980 msgr2=0x7f32381569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f323d400640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f32380d0980 0x7f32381569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f323d400640 1 -- 172.21.15.82:0/1519660427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f323815df90 msgr2=0x7f3238155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f323d400640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f323815df90 0x7f3238155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f323d400640 1 -- 172.21.15.82:0/1519660427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32280094d0 con 0x7f323815e370 2024-03-16T17:58:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f323ca00640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f32380d0980 0x7f32381569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f3237e00640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f323815df90 0x7f3238155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f323d400640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815e370 0x7f3238155e50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f323000c730 tx=0x7f323000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f3234c00640 1 -- 172.21.15.82:0/1519660427 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000ce40 con 0x7f323815e370 2024-03-16T17:58:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f3234c00640 1 -- 172.21.15.82:0/1519660427 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323000a640 con 0x7f323815e370 2024-03-16T17:58:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.104+0000 7f3234c00640 1 -- 172.21.15.82:0/1519660427 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000a9f0 con 0x7f323815e370 2024-03-16T17:58:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.105+0000 7f323f400640 1 -- 172.21.15.82:0/1519660427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3238147b30 con 0x7f323815e370 2024-03-16T17:58:43.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.105+0000 7f323f400640 1 -- 172.21.15.82:0/1519660427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32381480f0 con 0x7f323815e370 2024-03-16T17:58:43.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.106+0000 7f323f400640 1 -- 172.21.15.82:0/1519660427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3238067f30 con 0x7f323815e370 2024-03-16T17:58:43.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.107+0000 7f3234c00640 1 -- 172.21.15.82:0/1519660427 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f323000ab50 con 0x7f323815e370 2024-03-16T17:58:43.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.107+0000 7f3234c00640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f321406b3b0 0x7f321406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.107+0000 7f3234c00640 1 -- 172.21.15.82:0/1519660427 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3230088f70 con 0x7f323815e370 2024-03-16T17:58:43.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.108+0000 7f323ca00640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f321406b3b0 0x7f321406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:43.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.108+0000 7f323ca00640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f321406b3b0 0x7f321406d870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3228036000 tx=0x7f322803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:43.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.112+0000 7f3234c00640 1 -- 172.21.15.82:0/1519660427 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3230053fc0 con 0x7f323815e370 2024-03-16T17:58:43.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.378+0000 7f323f400640 1 -- 172.21.15.82:0/1519660427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f32380d13c0 con 0x7f323815e370 2024-03-16T17:58:43.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.382+0000 7f3234c00640 1 -- 172.21.15.82:0/1519660427 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f32300170a0 con 0x7f323815e370 2024-03-16T17:58:43.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 -- 172.21.15.82:0/1519660427 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f321406b3b0 msgr2=0x7f321406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f321406b3b0 0x7f321406d870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3228036000 tx=0x7f322803d040 comp rx=0 tx=0).stop 2024-03-16T17:58:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 -- 172.21.15.82:0/1519660427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815e370 msgr2=0x7f3238155e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815e370 0x7f3238155e50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f323000c730 tx=0x7f323000cbf0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 -- 172.21.15.82:0/1519660427 shutdown_connections 2024-03-16T17:58:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f321406b3b0 0x7f321406d870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f323815e370 0x7f3238155e50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f323815df90 0x7f3238155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 --2- 172.21.15.82:0/1519660427 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f32380d0980 0x7f32381569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 -- 172.21.15.82:0/1519660427 >> 172.21.15.82:0/1519660427 conn(0x7f323809d0e0 msgr2=0x7f32380b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 -- 172.21.15.82:0/1519660427 shutdown_connections 2024-03-16T17:58:43.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.385+0000 7f323f400640 1 -- 172.21.15.82:0/1519660427 wait complete. 2024-03-16T17:58:43.406 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-16T17:58:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.589+0000 7ff941e00640 1 Processor -- start 2024-03-16T17:58:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.589+0000 7ff941e00640 1 -- start start 2024-03-16T17:58:43.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.590+0000 7ff941e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff93c0d0e10 0x7ff93c0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.590+0000 7ff941e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff93c0d2410 0x7ff93c0b7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.590+0000 7ff941e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93c15e5b0 con 0x7ff93c15e1d0 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.590+0000 7ff941e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c0d28c0 con 0x7ff93c0d0e10 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.590+0000 7ff941e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93c0d2b60 con 0x7ff93c0d2410 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.591+0000 7ff93a000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff93c0d0e10 0x7ff93c0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.591+0000 7ff93a000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff93c0d0e10 0x7ff93c0d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37852/0 (socket says 172.21.15.82:37852) 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.591+0000 7ff93aa00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff93c15e1d0 0x7ff93c0d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35778/0 (socket says 172.21.15.82:35778) 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.591+0000 7ff93a000640 1 -- 172.21.15.82:0/735581703 learned_addr learned my addr 172.21.15.82:0/735581703 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.591+0000 7ff93b400640 1 --2- 172.21.15.82:0/735581703 >> v2:172.21.15.90:3300/0 conn(0x7ff93c0d2410 0x7ff93c0b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.591+0000 7ff93a000640 1 -- 172.21.15.82:0/735581703 >> v2:172.21.15.90:3300/0 conn(0x7ff93c0d2410 msgr2=0x7ff93c0b7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.591+0000 7ff93a000640 1 --2- 172.21.15.82:0/735581703 >> v2:172.21.15.90:3300/0 conn(0x7ff93c0d2410 0x7ff93c0b7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.591+0000 7ff93a000640 1 -- 172.21.15.82:0/735581703 >> v1:172.21.15.82:6789/0 conn(0x7ff93c15e1d0 legacy=0x7ff93c0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.591+0000 7ff93a000640 1 -- 172.21.15.82:0/735581703 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c0d2e40 con 0x7ff93c0d0e10 2024-03-16T17:58:43.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.591+0000 7ff93b400640 1 --2- 172.21.15.82:0/735581703 >> v2:172.21.15.90:3300/0 conn(0x7ff93c0d2410 0x7ff93c0b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.592+0000 7ff93a000640 1 --2- 172.21.15.82:0/735581703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 0x7ff93c0d1310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff924009e50 tx=0x7ff92402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d103b111526d51 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.592+0000 7ff939600640 1 -- 172.21.15.82:0/735581703 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92402cb90 con 0x7ff93c0d0e10 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.593+0000 7ff939600640 1 -- 172.21.15.82:0/735581703 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92402ccf0 con 0x7ff93c0d0e10 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.593+0000 7ff939600640 1 -- 172.21.15.82:0/735581703 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924034610 con 0x7ff93c0d0e10 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.593+0000 7ff941e00640 1 -- 172.21.15.82:0/735581703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 msgr2=0x7ff93c0d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.593+0000 7ff941e00640 1 --2- 172.21.15.82:0/735581703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 0x7ff93c0d1310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff924009e50 tx=0x7ff92402bff0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.593+0000 7ff941e00640 1 -- 172.21.15.82:0/735581703 shutdown_connections 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.593+0000 7ff941e00640 1 --2- 172.21.15.82:0/735581703 >> v2:172.21.15.90:3300/0 conn(0x7ff93c0d2410 0x7ff93c0b7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.593+0000 7ff941e00640 1 --2- 172.21.15.82:0/735581703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 0x7ff93c0d1310 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.593+0000 7ff941e00640 1 -- 172.21.15.82:0/735581703 >> 172.21.15.82:0/735581703 conn(0x7ff93c0d3e30 msgr2=0x7ff93c0d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.593+0000 7ff941e00640 1 -- 172.21.15.82:0/735581703 shutdown_connections 2024-03-16T17:58:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.593+0000 7ff941e00640 1 -- 172.21.15.82:0/735581703 wait complete. 2024-03-16T17:58:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.595+0000 7ff941e00640 1 Processor -- start 2024-03-16T17:58:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.595+0000 7ff941e00640 1 -- start start 2024-03-16T17:58:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.596+0000 7ff941e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 0x7ff93c0b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.596+0000 7ff941e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff93c0d2410 0x7ff93c0c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.596+0000 7ff941e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff93c15e1d0 0x7ff93c0cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.596+0000 7ff941e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0ccfe0 con 0x7ff93c0d0e10 2024-03-16T17:58:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.596+0000 7ff941e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0cc360 con 0x7ff93c15e1d0 2024-03-16T17:58:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.596+0000 7ff941e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93c0cc640 con 0x7ff93c0d2410 2024-03-16T17:58:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.596+0000 7ff93aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 0x7ff93c0b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.596+0000 7ff93aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 0x7ff93c0b43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37860/0 (socket says 172.21.15.82:37860) 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.596+0000 7ff93aa00640 1 -- 172.21.15.82:0/3069334540 learned_addr learned my addr 172.21.15.82:0/3069334540 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff93aa00640 1 -- 172.21.15.82:0/3069334540 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff93c0d2410 msgr2=0x7ff93c0c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff93aa00640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff93c0d2410 0x7ff93c0c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff93b400640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff93c15e1d0 0x7ff93c0cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff93aa00640 1 -- 172.21.15.82:0/3069334540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff93c15e1d0 msgr2=0x7ff93c0cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff93aa00640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff93c15e1d0 0x7ff93c0cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff93aa00640 1 -- 172.21.15.82:0/3069334540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff924009b30 con 0x7ff93c0d0e10 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff93aa00640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 0x7ff93c0b43d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00c3c0 tx=0x7ff92c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff933400640 1 -- 172.21.15.82:0/3069334540 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0074e0 con 0x7ff93c0d0e10 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff933400640 1 -- 172.21.15.82:0/3069334540 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c007640 con 0x7ff93c0d0e10 2024-03-16T17:58:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff933400640 1 -- 172.21.15.82:0/3069334540 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c004a20 con 0x7ff93c0d0e10 2024-03-16T17:58:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff941e00640 1 -- 172.21.15.82:0/3069334540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93c0cc920 con 0x7ff93c0d0e10 2024-03-16T17:58:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.597+0000 7ff941e00640 1 -- 172.21.15.82:0/3069334540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c0c8620 con 0x7ff93c0d0e10 2024-03-16T17:58:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.598+0000 7ff941e00640 1 -- 172.21.15.82:0/3069334540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93c068640 con 0x7ff93c0d0e10 2024-03-16T17:58:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.599+0000 7ff933400640 1 -- 172.21.15.82:0/3069334540 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7ff92c004b80 con 0x7ff93c0d0e10 2024-03-16T17:58:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.600+0000 7ff933400640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff91806b3b0 0x7ff91806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:43.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.600+0000 7ff933400640 1 -- 172.21.15.82:0/3069334540 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff92c089330 con 0x7ff93c0d0e10 2024-03-16T17:58:43.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.600+0000 7ff93a000640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff91806b3b0 0x7ff91806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:43.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.601+0000 7ff93a000640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff91806b3b0 0x7ff91806d870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff924035000 tx=0x7ff92402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:43.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.605+0000 7ff933400640 1 -- 172.21.15.82:0/3069334540 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff92c054240 con 0x7ff93c0d0e10 2024-03-16T17:58:43.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.875+0000 7ff941e00640 1 -- 172.21.15.82:0/3069334540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7ff93c0b8630 con 0x7ff93c0d0e10 2024-03-16T17:58:43.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.881+0000 7ff933400640 1 -- 172.21.15.82:0/3069334540 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff92c057900 con 0x7ff93c0d0e10 2024-03-16T17:58:43.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 -- 172.21.15.82:0/3069334540 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff91806b3b0 msgr2=0x7ff91806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff91806b3b0 0x7ff91806d870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff924035000 tx=0x7ff92402e000 comp rx=0 tx=0).stop 2024-03-16T17:58:43.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 -- 172.21.15.82:0/3069334540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 msgr2=0x7ff93c0b43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:43.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 0x7ff93c0b43d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00c3c0 tx=0x7ff92c00c880 comp rx=0 tx=0).stop 2024-03-16T17:58:43.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 -- 172.21.15.82:0/3069334540 shutdown_connections 2024-03-16T17:58:43.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff91806b3b0 0x7ff91806d870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff93c15e1d0 0x7ff93c0cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff93c0d2410 0x7ff93c0c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 --2- 172.21.15.82:0/3069334540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff93c0d0e10 0x7ff93c0b43d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 -- 172.21.15.82:0/3069334540 >> 172.21.15.82:0/3069334540 conn(0x7ff93c0d3e30 msgr2=0x7ff93c15d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 -- 172.21.15.82:0/3069334540 shutdown_connections 2024-03-16T17:58:43.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:43.885+0000 7ff941e00640 1 -- 172.21.15.82:0/3069334540 wait complete. 2024-03-16T17:58:43.910 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-16T17:58:44.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.115+0000 7fe0c9a00640 1 Processor -- start 2024-03-16T17:58:44.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.116+0000 7fe0c9a00640 1 -- start start 2024-03-16T17:58:44.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.117+0000 7fe0c9a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe0c415e180 0x7fe0c40cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.117+0000 7fe0c9a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 0x7fe0c40d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.117+0000 7fe0c9a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c415e560 con 0x7fe0c40d0ca0 2024-03-16T17:58:44.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.117+0000 7fe0c9a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c40d10d0 con 0x7fe0c415e180 2024-03-16T17:58:44.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.117+0000 7fe0c9a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40cf0d0 con 0x7fe0c415f110 2024-03-16T17:58:44.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.117+0000 7fe0c2a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe0c415e180 0x7fe0c40cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.117+0000 7fe0c3400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe0c40d0ca0 0x7fe0c4161c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35782/0 (socket says 172.21.15.82:35782) 2024-03-16T17:58:44.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.117+0000 7fe0c3400640 1 -- 172.21.15.82:0/1998773590 learned_addr learned my addr 172.21.15.82:0/1998773590 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:44.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.118+0000 7fe0c2000640 1 --2- 172.21.15.82:0/1998773590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 0x7fe0c40d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.118+0000 7fe0c2a00640 1 -- 172.21.15.82:0/1998773590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 msgr2=0x7fe0c40d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:44.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.118+0000 7fe0c2a00640 1 --2- 172.21.15.82:0/1998773590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 0x7fe0c40d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.118+0000 7fe0c2a00640 1 -- 172.21.15.82:0/1998773590 >> v1:172.21.15.82:6789/0 conn(0x7fe0c40d0ca0 legacy=0x7fe0c4161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.118+0000 7fe0c1600640 1 -- 172.21.15.82:0/1998773590 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215194628 0 0) 0x7fe0bc001110 con 0x7fe0c40d0ca0 2024-03-16T17:58:44.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.118+0000 7fe0c2a00640 1 -- 172.21.15.82:0/1998773590 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c40cf3b0 con 0x7fe0c415e180 2024-03-16T17:58:44.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.119+0000 7fe0c2a00640 1 --2- 172.21.15.82:0/1998773590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 0x7fe0c40cea30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009d90 tx=0x7fe0b402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedf579bd667ee41 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:44.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.119+0000 7fe0c1600640 1 -- 172.21.15.82:0/1998773590 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b402cc50 con 0x7fe0c415e180 2024-03-16T17:58:44.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.119+0000 7fe0c1600640 1 -- 172.21.15.82:0/1998773590 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b402cdb0 con 0x7fe0c415e180 2024-03-16T17:58:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.119+0000 7fe0c1600640 1 -- 172.21.15.82:0/1998773590 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4034660 con 0x7fe0c415e180 2024-03-16T17:58:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.120+0000 7fe0c9a00640 1 -- 172.21.15.82:0/1998773590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 msgr2=0x7fe0c40cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.120+0000 7fe0c9a00640 1 --2- 172.21.15.82:0/1998773590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 0x7fe0c40cea30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009d90 tx=0x7fe0b402bea0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.120+0000 7fe0c9a00640 1 -- 172.21.15.82:0/1998773590 shutdown_connections 2024-03-16T17:58:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.120+0000 7fe0c9a00640 1 --2- 172.21.15.82:0/1998773590 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 0x7fe0c40d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.120+0000 7fe0c9a00640 1 --2- 172.21.15.82:0/1998773590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 0x7fe0c40cea30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.120+0000 7fe0c9a00640 1 -- 172.21.15.82:0/1998773590 >> 172.21.15.82:0/1998773590 conn(0x7fe0c40d3e90 msgr2=0x7fe0c40d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.120+0000 7fe0c9a00640 1 -- 172.21.15.82:0/1998773590 shutdown_connections 2024-03-16T17:58:44.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.120+0000 7fe0c9a00640 1 -- 172.21.15.82:0/1998773590 wait complete. 2024-03-16T17:58:44.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.122+0000 7fe0c9a00640 1 Processor -- start 2024-03-16T17:58:44.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.122+0000 7fe0c9a00640 1 -- start start 2024-03-16T17:58:44.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.122+0000 7fe0c9a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c40d0ca0 0x7fe0c4158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c9a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 0x7fe0c41562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c9a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 0x7fe0c40cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c9a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40c7980 con 0x7fe0c415e180 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c9a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c4158560 con 0x7fe0c40d0ca0 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c9a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40cc270 con 0x7fe0c415f110 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 0x7fe0c41562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c2000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 0x7fe0c41562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37890/0 (socket says 172.21.15.82:37890) 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c2000640 1 -- 172.21.15.82:0/4151604111 learned_addr learned my addr 172.21.15.82:0/4151604111 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c2a00640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c40d0ca0 0x7fe0c4158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c3400640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 0x7fe0c40cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c2000640 1 -- 172.21.15.82:0/4151604111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 msgr2=0x7fe0c40cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c2000640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 0x7fe0c40cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c2000640 1 -- 172.21.15.82:0/4151604111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c40d0ca0 msgr2=0x7fe0c4158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c2000640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c40d0ca0 0x7fe0c4158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c2000640 1 -- 172.21.15.82:0/4151604111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b4009a70 con 0x7fe0c415e180 2024-03-16T17:58:44.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.123+0000 7fe0c3400640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 0x7fe0c40cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:44.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.124+0000 7fe0c2000640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 0x7fe0c41562c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000d5d0 tx=0x7fe0b000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:44.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.124+0000 7fe0baa00640 1 -- 172.21.15.82:0/4151604111 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00125f0 con 0x7fe0c415e180 2024-03-16T17:58:44.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.124+0000 7fe0baa00640 1 -- 172.21.15.82:0/4151604111 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b0012770 con 0x7fe0c415e180 2024-03-16T17:58:44.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.124+0000 7fe0baa00640 1 -- 172.21.15.82:0/4151604111 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0012b20 con 0x7fe0c415e180 2024-03-16T17:58:44.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.124+0000 7fe0c9a00640 1 -- 172.21.15.82:0/4151604111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c40c7bd0 con 0x7fe0c415e180 2024-03-16T17:58:44.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.124+0000 7fe0c9a00640 1 -- 172.21.15.82:0/4151604111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c40c8190 con 0x7fe0c415e180 2024-03-16T17:58:44.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.125+0000 7fe0c9a00640 1 -- 172.21.15.82:0/4151604111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c4067f30 con 0x7fe0c415e180 2024-03-16T17:58:44.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.126+0000 7fe0baa00640 1 -- 172.21.15.82:0/4151604111 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe0b0012c80 con 0x7fe0c415e180 2024-03-16T17:58:44.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.127+0000 7fe0baa00640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0a406b3b0 0x7fe0a406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.127+0000 7fe0baa00640 1 -- 172.21.15.82:0/4151604111 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe0b0089ea0 con 0x7fe0c415e180 2024-03-16T17:58:44.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.127+0000 7fe0c2a00640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0a406b3b0 0x7fe0a406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.128+0000 7fe0c2a00640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0a406b3b0 0x7fe0a406d870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4036000 tx=0x7fe0b4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:44.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.131+0000 7fe0baa00640 1 -- 172.21.15.82:0/4151604111 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe0b0054db0 con 0x7fe0c415e180 2024-03-16T17:58:44.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.401+0000 7fe0c9a00640 1 -- 172.21.15.82:0/4151604111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fe0c408d8f0 con 0x7fe0c415e180 2024-03-16T17:58:44.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.406+0000 7fe0baa00640 1 -- 172.21.15.82:0/4151604111 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe0b0058470 con 0x7fe0c415e180 2024-03-16T17:58:44.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.409+0000 7fe0c9a00640 1 -- 172.21.15.82:0/4151604111 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0a406b3b0 msgr2=0x7fe0a406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.409+0000 7fe0c9a00640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0a406b3b0 0x7fe0a406d870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4036000 tx=0x7fe0b4030000 comp rx=0 tx=0).stop 2024-03-16T17:58:44.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.409+0000 7fe0c9a00640 1 -- 172.21.15.82:0/4151604111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 msgr2=0x7fe0c41562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.409+0000 7fe0c9a00640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 0x7fe0c41562c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000d5d0 tx=0x7fe0b000da90 comp rx=0 tx=0).stop 2024-03-16T17:58:44.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.410+0000 7fe0c9a00640 1 -- 172.21.15.82:0/4151604111 shutdown_connections 2024-03-16T17:58:44.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.410+0000 7fe0c9a00640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0a406b3b0 0x7fe0a406d870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.410+0000 7fe0c9a00640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe0c415f110 0x7fe0c40cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.410+0000 7fe0c9a00640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe0c415e180 0x7fe0c41562c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.410+0000 7fe0c9a00640 1 --2- 172.21.15.82:0/4151604111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe0c40d0ca0 0x7fe0c4158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.410+0000 7fe0c9a00640 1 -- 172.21.15.82:0/4151604111 >> 172.21.15.82:0/4151604111 conn(0x7fe0c40d3e90 msgr2=0x7fe0c40b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:44.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.410+0000 7fe0c9a00640 1 -- 172.21.15.82:0/4151604111 shutdown_connections 2024-03-16T17:58:44.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.410+0000 7fe0c9a00640 1 -- 172.21.15.82:0/4151604111 wait complete. 2024-03-16T17:58:44.431 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-16T17:58:44.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.635+0000 7fea81400640 1 Processor -- start 2024-03-16T17:58:44.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.635+0000 7fea81400640 1 -- start start 2024-03-16T17:58:44.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.637+0000 7fea81400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fea7c145320 0x7fea7c165f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.637+0000 7fea81400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fea7c148120 0x7fea7c16c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.637+0000 7fea81400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7c148950 con 0x7fea7c166480 2024-03-16T17:58:44.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.637+0000 7fea81400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7c166910 con 0x7fea7c145320 2024-03-16T17:58:44.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.637+0000 7fea81400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c0b3480 con 0x7fea7c148120 2024-03-16T17:58:44.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.637+0000 7fea7a800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fea7c145320 0x7fea7c165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.637+0000 7fea7a800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fea7c145320 0x7fea7c165f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37904/0 (socket says 172.21.15.82:37904) 2024-03-16T17:58:44.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.637+0000 7fea7a800640 1 -- 172.21.15.82:0/3617732741 learned_addr learned my addr 172.21.15.82:0/3617732741 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:44.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.637+0000 7fea7b200640 1 --2- 172.21.15.82:0/3617732741 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fea7c148120 0x7fea7c16c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.638+0000 7fea7a800640 1 -- 172.21.15.82:0/3617732741 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fea7c148120 msgr2=0x7fea7c16c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:44.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.638+0000 7fea7a800640 1 --2- 172.21.15.82:0/3617732741 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fea7c148120 0x7fea7c16c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.638+0000 7fea7a800640 1 -- 172.21.15.82:0/3617732741 >> v1:172.21.15.90:6789/0 conn(0x7fea7c166480 legacy=0x7fea7c147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:44.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.638+0000 7fea7a800640 1 -- 172.21.15.82:0/3617732741 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c145700 con 0x7fea7c145320 2024-03-16T17:58:44.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.639+0000 7fea7a800640 1 --2- 172.21.15.82:0/3617732741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c145320 0x7fea7c165f40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009870 tx=0x7fea6c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f8ab1641716814 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:44.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.639+0000 7fea79400640 1 -- 172.21.15.82:0/3617732741 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c031640 con 0x7fea7c145320 2024-03-16T17:58:44.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.639+0000 7fea79400640 1 -- 172.21.15.82:0/3617732741 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c0317a0 con 0x7fea7c145320 2024-03-16T17:58:44.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.639+0000 7fea79400640 1 -- 172.21.15.82:0/3617732741 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c031b30 con 0x7fea7c145320 2024-03-16T17:58:44.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.640+0000 7fea81400640 1 -- 172.21.15.82:0/3617732741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c145320 msgr2=0x7fea7c165f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.640+0000 7fea81400640 1 --2- 172.21.15.82:0/3617732741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c145320 0x7fea7c165f40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fea6c009870 tx=0x7fea6c02b8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.640+0000 7fea81400640 1 -- 172.21.15.82:0/3617732741 shutdown_connections 2024-03-16T17:58:44.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.640+0000 7fea81400640 1 --2- 172.21.15.82:0/3617732741 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fea7c148120 0x7fea7c16c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.640+0000 7fea81400640 1 --2- 172.21.15.82:0/3617732741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c145320 0x7fea7c165f40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.640+0000 7fea81400640 1 -- 172.21.15.82:0/3617732741 >> 172.21.15.82:0/3617732741 conn(0x7fea7c06ae70 msgr2=0x7fea7c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:44.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.640+0000 7fea81400640 1 -- 172.21.15.82:0/3617732741 shutdown_connections 2024-03-16T17:58:44.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.640+0000 7fea81400640 1 -- 172.21.15.82:0/3617732741 wait complete. 2024-03-16T17:58:44.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.642+0000 7fea81400640 1 Processor -- start 2024-03-16T17:58:44.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.642+0000 7fea81400640 1 -- start start 2024-03-16T17:58:44.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea81400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c145320 0x7fea7c143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea81400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c148120 0x7fea7c161740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea81400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fea7c166480 0x7fea7c147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea81400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c1609d0 con 0x7fea7c148120 2024-03-16T17:58:44.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea81400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c15fdc0 con 0x7fea7c145320 2024-03-16T17:58:44.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea81400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c1600a0 con 0x7fea7c166480 2024-03-16T17:58:44.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea79e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c148120 0x7fea7c161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea79e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c148120 0x7fea7c161740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37906/0 (socket says 172.21.15.82:37906) 2024-03-16T17:58:44.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea79e00640 1 -- 172.21.15.82:0/240539066 learned_addr learned my addr 172.21.15.82:0/240539066 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:44.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea7a800640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c145320 0x7fea7c143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea79e00640 1 -- 172.21.15.82:0/240539066 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fea7c166480 msgr2=0x7fea7c147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea7b200640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fea7c166480 0x7fea7c147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.643+0000 7fea79e00640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fea7c166480 0x7fea7c147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.644+0000 7fea79e00640 1 -- 172.21.15.82:0/240539066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c145320 msgr2=0x7fea7c143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.644+0000 7fea79e00640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c145320 0x7fea7c143950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.644+0000 7fea79e00640 1 -- 172.21.15.82:0/240539066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c0094d0 con 0x7fea7c148120 2024-03-16T17:58:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.644+0000 7fea79e00640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c148120 0x7fea7c161740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea6800c410 tx=0x7fea6800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.644+0000 7fea72a00640 1 -- 172.21.15.82:0/240539066 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea680035f0 con 0x7fea7c148120 2024-03-16T17:58:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.644+0000 7fea72a00640 1 -- 172.21.15.82:0/240539066 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea68003750 con 0x7fea7c148120 2024-03-16T17:58:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.644+0000 7fea72a00640 1 -- 172.21.15.82:0/240539066 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea68003b40 con 0x7fea7c148120 2024-03-16T17:58:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.644+0000 7fea81400640 1 -- 172.21.15.82:0/240539066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7c160380 con 0x7fea7c148120 2024-03-16T17:58:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.644+0000 7fea81400640 1 -- 172.21.15.82:0/240539066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c143e90 con 0x7fea7c148120 2024-03-16T17:58:44.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.645+0000 7fea81400640 1 -- 172.21.15.82:0/240539066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7c0b38f0 con 0x7fea7c148120 2024-03-16T17:58:44.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.646+0000 7fea72a00640 1 -- 172.21.15.82:0/240539066 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fea68003ca0 con 0x7fea7c148120 2024-03-16T17:58:44.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.647+0000 7fea72a00640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fea5c06b3b0 0x7fea5c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:44.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.647+0000 7fea72a00640 1 -- 172.21.15.82:0/240539066 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fea68088fa0 con 0x7fea7c148120 2024-03-16T17:58:44.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.647+0000 7fea7a800640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fea5c06b3b0 0x7fea5c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:44.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.648+0000 7fea7a800640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fea5c06b3b0 0x7fea5c06d870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea6c036000 tx=0x7fea6c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:44.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.651+0000 7fea72a00640 1 -- 172.21.15.82:0/240539066 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fea7c0b38f0 con 0x7fea7c148120 2024-03-16T17:58:44.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.916+0000 7fea81400640 1 -- 172.21.15.82:0/240539066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fea7c0b38f0 con 0x7fea7c148120 2024-03-16T17:58:44.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.920+0000 7fea72a00640 1 -- 172.21.15.82:0/240539066 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fea7c0b38f0 con 0x7fea7c148120 2024-03-16T17:58:44.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.922+0000 7fea81400640 1 -- 172.21.15.82:0/240539066 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fea5c06b3b0 msgr2=0x7fea5c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.922+0000 7fea81400640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fea5c06b3b0 0x7fea5c06d870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea6c036000 tx=0x7fea6c03c040 comp rx=0 tx=0).stop 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.922+0000 7fea81400640 1 -- 172.21.15.82:0/240539066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c148120 msgr2=0x7fea7c161740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.922+0000 7fea81400640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c148120 0x7fea7c161740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fea6800c410 tx=0x7fea6800c8d0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.923+0000 7fea81400640 1 -- 172.21.15.82:0/240539066 shutdown_connections 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.923+0000 7fea81400640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fea5c06b3b0 0x7fea5c06d870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.923+0000 7fea81400640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fea7c166480 0x7fea7c147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.923+0000 7fea81400640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea7c148120 0x7fea7c161740 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.923+0000 7fea81400640 1 --2- 172.21.15.82:0/240539066 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c145320 0x7fea7c143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.923+0000 7fea81400640 1 -- 172.21.15.82:0/240539066 >> 172.21.15.82:0/240539066 conn(0x7fea7c06ae70 msgr2=0x7fea7c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.923+0000 7fea81400640 1 -- 172.21.15.82:0/240539066 shutdown_connections 2024-03-16T17:58:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:44.923+0000 7fea81400640 1 -- 172.21.15.82:0/240539066 wait complete. 2024-03-16T17:58:44.943 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-16T17:58:45.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.147+0000 7f9eb2200640 1 Processor -- start 2024-03-16T17:58:45.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.147+0000 7f9eb2200640 1 -- start start 2024-03-16T17:58:45.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.149+0000 7f9eb2200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9eac06c870 0x7f9eac15f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.149+0000 7f9eb2200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9eac0ce820 0x7f9eac0cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.149+0000 7f9eb2200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac0cddd0 con 0x7f9eac06d410 2024-03-16T17:58:45.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.149+0000 7f9eb2200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac15f570 con 0x7f9eac06c870 2024-03-16T17:58:45.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.149+0000 7f9eb2200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac15bf80 con 0x7f9eac0ce820 2024-03-16T17:58:45.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.149+0000 7f9eab400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9eac06c870 0x7f9eac15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.149+0000 7f9eab400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9eac06c870 0x7f9eac15f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41882/0 (socket says 172.21.15.82:41882) 2024-03-16T17:58:45.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.149+0000 7f9eab400640 1 -- 172.21.15.82:0/297872934 learned_addr learned my addr 172.21.15.82:0/297872934 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:45.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.150+0000 7f9eabe00640 1 --2- 172.21.15.82:0/297872934 >> v2:172.21.15.90:3300/0 conn(0x7f9eac0ce820 0x7f9eac0cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.150+0000 7f9eab400640 1 -- 172.21.15.82:0/297872934 >> v2:172.21.15.90:3300/0 conn(0x7f9eac0ce820 msgr2=0x7f9eac0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:45.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.150+0000 7f9eab400640 1 --2- 172.21.15.82:0/297872934 >> v2:172.21.15.90:3300/0 conn(0x7f9eac0ce820 0x7f9eac0cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.150+0000 7f9eab400640 1 -- 172.21.15.82:0/297872934 >> v1:172.21.15.82:6789/0 conn(0x7f9eac06d410 legacy=0x7f9eac06cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.150+0000 7f9eab400640 1 -- 172.21.15.82:0/297872934 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac0b4690 con 0x7f9eac06c870 2024-03-16T17:58:45.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.150+0000 7f9eaa000640 1 -- 172.21.15.82:0/297872934 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408243730 0 0) 0x7f9eac0cddd0 con 0x7f9eac06d410 2024-03-16T17:58:45.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.151+0000 7f9eab400640 1 --2- 172.21.15.82:0/297872934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06c870 0x7f9eac15f030 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9eac06c460 tx=0x7f9ea402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52be5ba75423226 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:45.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.151+0000 7f9eaa000640 1 -- 172.21.15.82:0/297872934 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea402ca80 con 0x7f9eac06c870 2024-03-16T17:58:45.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.151+0000 7f9eaa000640 1 -- 172.21.15.82:0/297872934 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea402cbe0 con 0x7f9eac06c870 2024-03-16T17:58:45.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.151+0000 7f9eaa000640 1 -- 172.21.15.82:0/297872934 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea40343f0 con 0x7f9eac06c870 2024-03-16T17:58:45.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.152+0000 7f9eb2200640 1 -- 172.21.15.82:0/297872934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06c870 msgr2=0x7f9eac15f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.152+0000 7f9eb2200640 1 --2- 172.21.15.82:0/297872934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06c870 0x7f9eac15f030 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9eac06c460 tx=0x7f9ea402bee0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.152+0000 7f9eb2200640 1 -- 172.21.15.82:0/297872934 shutdown_connections 2024-03-16T17:58:45.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.152+0000 7f9eb2200640 1 --2- 172.21.15.82:0/297872934 >> v2:172.21.15.90:3300/0 conn(0x7f9eac0ce820 0x7f9eac0cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.152+0000 7f9eb2200640 1 --2- 172.21.15.82:0/297872934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06c870 0x7f9eac15f030 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.152+0000 7f9eb2200640 1 -- 172.21.15.82:0/297872934 >> 172.21.15.82:0/297872934 conn(0x7f9eac0b17b0 msgr2=0x7f9eac0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:45.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.152+0000 7f9eb2200640 1 -- 172.21.15.82:0/297872934 shutdown_connections 2024-03-16T17:58:45.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.152+0000 7f9eb2200640 1 -- 172.21.15.82:0/297872934 wait complete. 2024-03-16T17:58:45.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.154+0000 7f9eb2200640 1 Processor -- start 2024-03-16T17:58:45.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.154+0000 7f9eb2200640 1 -- start start 2024-03-16T17:58:45.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.155+0000 7f9eb2200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9eac06c870 0x7f9eac15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.155+0000 7f9eb2200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06d410 0x7f9eac14af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.155+0000 7f9eb2200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eac0ce820 0x7f9eac14b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.155+0000 7f9eb2200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac159200 con 0x7f9eac06d410 2024-03-16T17:58:45.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.155+0000 7f9eb2200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0b0300 con 0x7f9eac0ce820 2024-03-16T17:58:45.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.155+0000 7f9eb2200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac0b05e0 con 0x7f9eac06c870 2024-03-16T17:58:45.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.155+0000 7f9eaaa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06d410 0x7f9eac14af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.155+0000 7f9eaaa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06d410 0x7f9eac14af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41888/0 (socket says 172.21.15.82:41888) 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.155+0000 7f9eaaa00640 1 -- 172.21.15.82:0/3907056677 learned_addr learned my addr 172.21.15.82:0/3907056677 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eab400640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9eac06c870 0x7f9eac15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eabe00640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eac0ce820 0x7f9eac14b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eaaa00640 1 -- 172.21.15.82:0/3907056677 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9eac06c870 msgr2=0x7f9eac15a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eaaa00640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9eac06c870 0x7f9eac15a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eaaa00640 1 -- 172.21.15.82:0/3907056677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eac0ce820 msgr2=0x7f9eac14b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eaaa00640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eac0ce820 0x7f9eac14b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eaaa00640 1 -- 172.21.15.82:0/3907056677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4009b30 con 0x7f9eac06d410 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eabe00640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eac0ce820 0x7f9eac14b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eaaa00640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06d410 0x7f9eac14af90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c410 tx=0x7f9e9400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9ea3400640 1 -- 172.21.15.82:0/3907056677 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e940035f0 con 0x7f9eac06d410 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9ea3400640 1 -- 172.21.15.82:0/3907056677 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e94003770 con 0x7f9eac06d410 2024-03-16T17:58:45.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9ea3400640 1 -- 172.21.15.82:0/3907056677 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94003b20 con 0x7f9eac06d410 2024-03-16T17:58:45.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eb2200640 1 -- 172.21.15.82:0/3907056677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac0b08c0 con 0x7f9eac06d410 2024-03-16T17:58:45.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.156+0000 7f9eb2200640 1 -- 172.21.15.82:0/3907056677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac0b0e80 con 0x7f9eac06d410 2024-03-16T17:58:45.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.158+0000 7f9eb2200640 1 -- 172.21.15.82:0/3907056677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac0b10a0 con 0x7f9eac06d410 2024-03-16T17:58:45.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.158+0000 7f9ea3400640 1 -- 172.21.15.82:0/3907056677 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f9e94003c80 con 0x7f9eac06d410 2024-03-16T17:58:45.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.159+0000 7f9ea3400640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e8806b3b0 0x7f9e8806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.159+0000 7f9ea3400640 1 -- 172.21.15.82:0/3907056677 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9e94088f20 con 0x7f9eac06d410 2024-03-16T17:58:45.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.159+0000 7f9eab400640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e8806b3b0 0x7f9e8806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.160+0000 7f9eab400640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e8806b3b0 0x7f9e8806d870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4035000 tx=0x7f9ea4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:45.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.164+0000 7f9ea3400640 1 -- 172.21.15.82:0/3907056677 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e94053ed0 con 0x7f9eac06d410 2024-03-16T17:58:45.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.428+0000 7f9eb2200640 1 -- 172.21.15.82:0/3907056677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f9eac073420 con 0x7f9eac06d410 2024-03-16T17:58:45.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.434+0000 7f9ea3400640 1 -- 172.21.15.82:0/3907056677 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9e940170a0 con 0x7f9eac06d410 2024-03-16T17:58:45.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 -- 172.21.15.82:0/3907056677 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e8806b3b0 msgr2=0x7f9e8806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e8806b3b0 0x7f9e8806d870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4035000 tx=0x7f9ea4030000 comp rx=0 tx=0).stop 2024-03-16T17:58:45.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 -- 172.21.15.82:0/3907056677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06d410 msgr2=0x7f9eac14af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06d410 0x7f9eac14af90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c410 tx=0x7f9e9400c8d0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 -- 172.21.15.82:0/3907056677 shutdown_connections 2024-03-16T17:58:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9e8806b3b0 0x7f9e8806d870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9eac0ce820 0x7f9eac14b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9eac06d410 0x7f9eac14af90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 --2- 172.21.15.82:0/3907056677 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9eac06c870 0x7f9eac15a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 -- 172.21.15.82:0/3907056677 >> 172.21.15.82:0/3907056677 conn(0x7f9eac0b17b0 msgr2=0x7f9eac0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 -- 172.21.15.82:0/3907056677 shutdown_connections 2024-03-16T17:58:45.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.437+0000 7f9eb2200640 1 -- 172.21.15.82:0/3907056677 wait complete. 2024-03-16T17:58:45.458 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-16T17:58:45.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.664+0000 7fa7d3e00640 1 Processor -- start 2024-03-16T17:58:45.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.665+0000 7fa7d3e00640 1 -- start start 2024-03-16T17:58:45.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.666+0000 7fa7d3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa7d40cdca0 0x7fa7d406c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.666+0000 7fa7d3e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d40b3150 con 0x7fa7d406cdc0 2024-03-16T17:58:45.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.666+0000 7fa7d3e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d40b2b40 con 0x7fa7d40b3cf0 2024-03-16T17:58:45.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.666+0000 7fa7d3e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d4070e80 con 0x7fa7d40cdca0 2024-03-16T17:58:45.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.667+0000 7fa7ca000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa7d406cdc0 0x7fa7d40b35e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60238/0 (socket says 172.21.15.82:60238) 2024-03-16T17:58:45.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.667+0000 7fa7ca000640 1 -- 172.21.15.82:0/2207022757 learned_addr learned my addr 172.21.15.82:0/2207022757 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:45.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.667+0000 7fa7d2a00640 1 --2- 172.21.15.82:0/2207022757 >> v2:172.21.15.90:3300/0 conn(0x7fa7d40cdca0 0x7fa7d406c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.667+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693340520 0 0) 0x7fa7d40b3150 con 0x7fa7d406cdc0 2024-03-16T17:58:45.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.667+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7b8003520 con 0x7fa7d406cdc0 2024-03-16T17:58:45.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.667+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2814721408 0 0) 0x7fa7b8003520 con 0x7fa7d406cdc0 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 >> v2:172.21.15.90:3300/0 conn(0x7fa7d40cdca0 msgr2=0x7fa7d406c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 --2- 172.21.15.82:0/2207022757 >> v2:172.21.15.90:3300/0 conn(0x7fa7d40cdca0 0x7fa7d406c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 >> v1:172.21.15.90:6789/0 conn(0x7fa7d40b3cf0 legacy=0x7fa7d40ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d40708c0 con 0x7fa7d406cdc0 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fa7bc001670 con 0x7fa7d406cdc0 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 >> v1:172.21.15.82:6789/0 conn(0x7fa7d406cdc0 legacy=0x7fa7d40b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b8003a30 0x7fa7b8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa7b80047c0 0x7fa7b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7b80051d0 0x7fa7b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4070e80 con 0x7fa7b8003a30 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b800c100 con 0x7fa7b80051d0 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b800c2c0 con 0x7fa7b80047c0 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2a00640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b8003a30 0x7fa7b8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d3400640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7b80051d0 0x7fa7b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7ca000640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa7b80047c0 0x7fa7b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2a00640 1 -- 172.21.15.82:0/2207022757 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa7b80047c0 msgr2=0x7fa7b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:45.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2a00640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa7b80047c0 0x7fa7b8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2a00640 1 -- 172.21.15.82:0/2207022757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7b80051d0 msgr2=0x7fa7b800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:45.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2a00640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7b80051d0 0x7fa7b800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d3400640 1 -- 172.21.15.82:0/2207022757 reap_dead start 2024-03-16T17:58:45.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.668+0000 7fa7d2a00640 1 -- 172.21.15.82:0/2207022757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b8003520 con 0x7fa7b8003a30 2024-03-16T17:58:45.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.670+0000 7fa7d2a00640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b8003a30 0x7fa7b8003e50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400aa40 tx=0x7fa7b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8f77a915b5ca7a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:45.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.670+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c402cbb0 con 0x7fa7b8003a30 2024-03-16T17:58:45.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.670+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c402cd10 con 0x7fa7b8003a30 2024-03-16T17:58:45.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.670+0000 7fa7d2000640 1 -- 172.21.15.82:0/2207022757 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4034660 con 0x7fa7b8003a30 2024-03-16T17:58:45.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.670+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2207022757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b8003a30 msgr2=0x7fa7b8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.670+0000 7fa7d3e00640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b8003a30 0x7fa7b8003e50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa7c400aa40 tx=0x7fa7b80057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.670+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2207022757 shutdown_connections 2024-03-16T17:58:45.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.670+0000 7fa7d3e00640 1 --2- 172.21.15.82:0/2207022757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b8003a30 0x7fa7b8003e50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.670+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2207022757 >> 172.21.15.82:0/2207022757 conn(0x7fa7d406ae70 msgr2=0x7fa7d406b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:45.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.671+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2207022757 shutdown_connections 2024-03-16T17:58:45.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.671+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2207022757 wait complete. 2024-03-16T17:58:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.672+0000 7fa7d3e00640 1 Processor -- start 2024-03-16T17:58:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.672+0000 7fa7d3e00640 1 -- start start 2024-03-16T17:58:45.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.673+0000 7fa7d3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7b8003a30 0x7fa7d41f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.673+0000 7fa7d3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d40cac40 0x7fa7d41513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.673+0000 7fa7d3e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa7d40cb0c0 0x7fa7d41518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.673+0000 7fa7d3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4151e30 con 0x7fa7d40cac40 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.673+0000 7fa7d3e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d414d920 con 0x7fa7b8003a30 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.673+0000 7fa7d3e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40c9f20 con 0x7fa7d40cb0c0 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.674+0000 7fa7ca000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d40cac40 0x7fa7d41513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.674+0000 7fa7ca000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d40cac40 0x7fa7d41513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41906/0 (socket says 172.21.15.82:41906) 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.674+0000 7fa7ca000640 1 -- 172.21.15.82:0/2988274201 learned_addr learned my addr 172.21.15.82:0/2988274201 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.674+0000 7fa7d2a00640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7b8003a30 0x7fa7d41f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.674+0000 7fa7ca000640 1 -- 172.21.15.82:0/2988274201 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa7d40cb0c0 msgr2=0x7fa7d41518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.674+0000 7fa7ca000640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa7d40cb0c0 0x7fa7d41518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.674+0000 7fa7ca000640 1 -- 172.21.15.82:0/2988274201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7b8003a30 msgr2=0x7fa7d41f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.674+0000 7fa7ca000640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7b8003a30 0x7fa7d41f4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.674+0000 7fa7ca000640 1 -- 172.21.15.82:0/2988274201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d41f4dc0 con 0x7fa7d40cac40 2024-03-16T17:58:45.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.674+0000 7fa7ca000640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d40cac40 0x7fa7d41513b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00d7d0 tx=0x7fa7bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.675+0000 7fa7cbe00640 1 -- 172.21.15.82:0/2988274201 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0072d0 con 0x7fa7d40cac40 2024-03-16T17:58:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.675+0000 7fa7cbe00640 1 -- 172.21.15.82:0/2988274201 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc007430 con 0x7fa7d40cac40 2024-03-16T17:58:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.675+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2988274201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d40c8be0 con 0x7fa7d40cac40 2024-03-16T17:58:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.675+0000 7fa7cbe00640 1 -- 172.21.15.82:0/2988274201 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0047e0 con 0x7fa7d40cac40 2024-03-16T17:58:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.675+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2988274201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d40c9440 con 0x7fa7d40cac40 2024-03-16T17:58:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.676+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2988274201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d404f250 con 0x7fa7d40cac40 2024-03-16T17:58:45.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.677+0000 7fa7cbe00640 1 -- 172.21.15.82:0/2988274201 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa7bc004940 con 0x7fa7d40cac40 2024-03-16T17:58:45.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.678+0000 7fa7cbe00640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa7b006b3b0 0x7fa7b006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:45.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.678+0000 7fa7cbe00640 1 -- 172.21.15.82:0/2988274201 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa7bc089480 con 0x7fa7d40cac40 2024-03-16T17:58:45.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.678+0000 7fa7d2a00640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa7b006b3b0 0x7fa7b006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:45.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.679+0000 7fa7d2a00640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa7b006b3b0 0x7fa7b006d870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4036000 tx=0x7fa7c4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:45.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.682+0000 7fa7cbe00640 1 -- 172.21.15.82:0/2988274201 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa7bc054390 con 0x7fa7d40cac40 2024-03-16T17:58:45.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.948+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2988274201 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa7d4068240 con 0x7fa7d40cac40 2024-03-16T17:58:45.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.953+0000 7fa7cbe00640 1 -- 172.21.15.82:0/2988274201 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa7bc0190a0 con 0x7fa7d40cac40 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.955+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2988274201 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa7b006b3b0 msgr2=0x7fa7b006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.955+0000 7fa7d3e00640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa7b006b3b0 0x7fa7b006d870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4036000 tx=0x7fa7c4030000 comp rx=0 tx=0).stop 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.955+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2988274201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d40cac40 msgr2=0x7fa7d41513b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.955+0000 7fa7d3e00640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d40cac40 0x7fa7d41513b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00d7d0 tx=0x7fa7bc00dc90 comp rx=0 tx=0).stop 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.956+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2988274201 shutdown_connections 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.956+0000 7fa7d3e00640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa7b006b3b0 0x7fa7b006d870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.956+0000 7fa7d3e00640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa7d40cb0c0 0x7fa7d41518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.956+0000 7fa7d3e00640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7d40cac40 0x7fa7d41513b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.956+0000 7fa7d3e00640 1 --2- 172.21.15.82:0/2988274201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7b8003a30 0x7fa7d41f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.956+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2988274201 >> 172.21.15.82:0/2988274201 conn(0x7fa7d406ae70 msgr2=0x7fa7d4071450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.956+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2988274201 shutdown_connections 2024-03-16T17:58:45.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:45.956+0000 7fa7d3e00640 1 -- 172.21.15.82:0/2988274201 wait complete. 2024-03-16T17:58:45.979 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:46.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.185+0000 7f9445600640 1 Processor -- start 2024-03-16T17:58:46.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.186+0000 7f9445600640 1 -- start start 2024-03-16T17:58:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.187+0000 7f9445600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f944015e180 0x7f944006c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.187+0000 7f9445600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f94400b4430 0x7f944006c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.187+0000 7f9445600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944006ced0 0x7f9440161c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.187+0000 7f9445600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944006d300 con 0x7f944015e180 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.187+0000 7f9445600640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944015e560 con 0x7f94400b4430 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.187+0000 7f9445600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94400d01b0 con 0x7f944006ced0 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.187+0000 7f943ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f944015e180 0x7f944006c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.187+0000 7f943ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f944015e180 0x7f944006c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41922/0 (socket says 172.21.15.82:41922) 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.187+0000 7f943ea00640 1 -- 172.21.15.82:0/3191199551 learned_addr learned my addr 172.21.15.82:0/3191199551 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.188+0000 7f943e000640 1 --2- 172.21.15.82:0/3191199551 >> v2:172.21.15.90:3300/0 conn(0x7f94400b4430 0x7f944006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.188+0000 7f943f400640 1 --2- 172.21.15.82:0/3191199551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944006ced0 0x7f9440161c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.188+0000 7f943ea00640 1 -- 172.21.15.82:0/3191199551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944006ced0 msgr2=0x7f9440161c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.188+0000 7f943ea00640 1 --2- 172.21.15.82:0/3191199551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944006ced0 0x7f9440161c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.188+0000 7f943ea00640 1 -- 172.21.15.82:0/3191199551 >> v2:172.21.15.90:3300/0 conn(0x7f94400b4430 msgr2=0x7f944006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.188+0000 7f943ea00640 1 --2- 172.21.15.82:0/3191199551 >> v2:172.21.15.90:3300/0 conn(0x7f94400b4430 0x7f944006c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.188+0000 7f943ea00640 1 -- 172.21.15.82:0/3191199551 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94400d0490 con 0x7f944015e180 2024-03-16T17:58:46.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.189+0000 7f943ea00640 1 --2- 172.21.15.82:0/3191199551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944015e180 0x7f944006c190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9440071cf0 tx=0x7f942802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c83adb4a0273c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:46.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.189+0000 7f943d600640 1 -- 172.21.15.82:0/3191199551 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428031870 con 0x7f944015e180 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.189+0000 7f943d600640 1 -- 172.21.15.82:0/3191199551 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94280319d0 con 0x7f944015e180 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.189+0000 7f943d600640 1 -- 172.21.15.82:0/3191199551 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428031d60 con 0x7f944015e180 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.190+0000 7f9445600640 1 -- 172.21.15.82:0/3191199551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944015e180 msgr2=0x7f944006c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.190+0000 7f9445600640 1 --2- 172.21.15.82:0/3191199551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944015e180 0x7f944006c190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9440071cf0 tx=0x7f942802b910 comp rx=0 tx=0).stop 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.190+0000 7f9445600640 1 -- 172.21.15.82:0/3191199551 shutdown_connections 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.190+0000 7f9445600640 1 --2- 172.21.15.82:0/3191199551 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944006ced0 0x7f9440161c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.190+0000 7f9445600640 1 --2- 172.21.15.82:0/3191199551 >> v2:172.21.15.90:3300/0 conn(0x7f94400b4430 0x7f944006c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.190+0000 7f9445600640 1 --2- 172.21.15.82:0/3191199551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944015e180 0x7f944006c190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.190+0000 7f9445600640 1 -- 172.21.15.82:0/3191199551 >> 172.21.15.82:0/3191199551 conn(0x7f94400b37f0 msgr2=0x7f94400b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.190+0000 7f9445600640 1 -- 172.21.15.82:0/3191199551 shutdown_connections 2024-03-16T17:58:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.190+0000 7f9445600640 1 -- 172.21.15.82:0/3191199551 wait complete. 2024-03-16T17:58:46.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.192+0000 7f9445600640 1 Processor -- start 2024-03-16T17:58:46.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.192+0000 7f9445600640 1 -- start start 2024-03-16T17:58:46.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.193+0000 7f9445600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f944006ced0 0x7f944015b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.193+0000 7f9445600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944015e180 0x7f9440159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.193+0000 7f9445600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944009f580 0x7f944009bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.193+0000 7f9445600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944015c760 con 0x7f944009f580 2024-03-16T17:58:46.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.193+0000 7f9445600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944015a970 con 0x7f944006ced0 2024-03-16T17:58:46.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.193+0000 7f9445600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944015a4c0 con 0x7f944015e180 2024-03-16T17:58:46.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.193+0000 7f943f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944009f580 0x7f944009bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.193+0000 7f943f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944009f580 0x7f944009bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41928/0 (socket says 172.21.15.82:41928) 2024-03-16T17:58:46.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.193+0000 7f943f400640 1 -- 172.21.15.82:0/1406997796 learned_addr learned my addr 172.21.15.82:0/1406997796 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:46.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.194+0000 7f943ea00640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f944006ced0 0x7f944015b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.194+0000 7f943e000640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944015e180 0x7f9440159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.194+0000 7f943f400640 1 -- 172.21.15.82:0/1406997796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944015e180 msgr2=0x7f9440159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.194+0000 7f943f400640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944015e180 0x7f9440159f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.194+0000 7f943f400640 1 -- 172.21.15.82:0/1406997796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f944006ced0 msgr2=0x7f944015b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.194+0000 7f943f400640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f944006ced0 0x7f944015b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.194+0000 7f943f400640 1 -- 172.21.15.82:0/1406997796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94280094d0 con 0x7f944009f580 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.194+0000 7f943f400640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944009f580 0x7f944009bae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f943800c550 tx=0x7f943800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.194+0000 7f9436a00640 1 -- 172.21.15.82:0/1406997796 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438004220 con 0x7f944009f580 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.195+0000 7f9436a00640 1 -- 172.21.15.82:0/1406997796 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9438004380 con 0x7f944009f580 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.195+0000 7f9445600640 1 -- 172.21.15.82:0/1406997796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944015bec0 con 0x7f944009f580 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.195+0000 7f9436a00640 1 -- 172.21.15.82:0/1406997796 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438003830 con 0x7f944009f580 2024-03-16T17:58:46.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.195+0000 7f9445600640 1 -- 172.21.15.82:0/1406997796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944015c480 con 0x7f944009f580 2024-03-16T17:58:46.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.196+0000 7f9445600640 1 -- 172.21.15.82:0/1406997796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944006b9a0 con 0x7f944009f580 2024-03-16T17:58:46.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.197+0000 7f9436a00640 1 -- 172.21.15.82:0/1406997796 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f9438003990 con 0x7f944009f580 2024-03-16T17:58:46.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.198+0000 7f9436a00640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f941c06b3e0 0x7f941c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.198+0000 7f9436a00640 1 -- 172.21.15.82:0/1406997796 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f94380896b0 con 0x7f944009f580 2024-03-16T17:58:46.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.198+0000 7f943ea00640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f941c06b3e0 0x7f941c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.199+0000 7f943ea00640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f941c06b3e0 0x7f941c06d8a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9428036000 tx=0x7f942803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:46.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.203+0000 7f9436a00640 1 -- 172.21.15.82:0/1406997796 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9438054530 con 0x7f944009f580 2024-03-16T17:58:46.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.401+0000 7f9445600640 1 -- 172.21.15.82:0/1406997796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9440070aa0 con 0x7f944009f580 2024-03-16T17:58:46.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.403+0000 7f9436a00640 1 -- 172.21.15.82:0/1406997796 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12855 (secure 0 0 0) 0x7f943801e020 con 0x7f944009f580 2024-03-16T17:58:46.407 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:46.407 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":30,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:42.382483+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.407+0000 7f9445600640 1 -- 172.21.15.82:0/1406997796 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f941c06b3e0 msgr2=0x7f941c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.407+0000 7f9445600640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f941c06b3e0 0x7f941c06d8a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9428036000 tx=0x7f942803c040 comp rx=0 tx=0).stop 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.407+0000 7f9445600640 1 -- 172.21.15.82:0/1406997796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944009f580 msgr2=0x7f944009bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.407+0000 7f9445600640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944009f580 0x7f944009bae0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f943800c550 tx=0x7f943800ca10 comp rx=0 tx=0).stop 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.408+0000 7f9445600640 1 -- 172.21.15.82:0/1406997796 shutdown_connections 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.408+0000 7f9445600640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f941c06b3e0 0x7f941c06d8a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.408+0000 7f9445600640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f944009f580 0x7f944009bae0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.408+0000 7f9445600640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f944015e180 0x7f9440159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.408+0000 7f9445600640 1 --2- 172.21.15.82:0/1406997796 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f944006ced0 0x7f944015b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.408+0000 7f9445600640 1 -- 172.21.15.82:0/1406997796 >> 172.21.15.82:0/1406997796 conn(0x7f94400b37f0 msgr2=0x7f94400d24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:46.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.408+0000 7f9445600640 1 -- 172.21.15.82:0/1406997796 shutdown_connections 2024-03-16T17:58:46.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.408+0000 7f9445600640 1 -- 172.21.15.82:0/1406997796 wait complete. 2024-03-16T17:58:46.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T17:58:46.432 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T17:58:46.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.629+0000 7fa245600640 1 Processor -- start 2024-03-16T17:58:46.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.629+0000 7fa245600640 1 -- start start 2024-03-16T17:58:46.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.630+0000 7fa245600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa2401469c0 0x7fa240162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.630+0000 7fa245600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 0x7fa2400ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.631+0000 7fa245600640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2400ce760 con 0x7fa240147060 2024-03-16T17:58:46.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.631+0000 7fa245600640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa240146da0 con 0x7fa2401469c0 2024-03-16T17:58:46.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.631+0000 7fa245600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2401628a0 con 0x7fa2400cddc0 2024-03-16T17:58:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.631+0000 7fa23ea00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa2401469c0 0x7fa240162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.631+0000 7fa23f400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 0x7fa2400ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.631+0000 7fa23f400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 0x7fa2400ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:40556/0 (socket says 172.21.15.82:40556) 2024-03-16T17:58:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.631+0000 7fa23f400640 1 -- 172.21.15.82:0/2586511878 learned_addr learned my addr 172.21.15.82:0/2586511878 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.632+0000 7fa23f400640 1 -- 172.21.15.82:0/2586511878 >> v2:172.21.15.90:3300/0 conn(0x7fa2401469c0 msgr2=0x7fa240162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.632+0000 7fa23f400640 1 --2- 172.21.15.82:0/2586511878 >> v2:172.21.15.90:3300/0 conn(0x7fa2401469c0 0x7fa240162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.632+0000 7fa23f400640 1 -- 172.21.15.82:0/2586511878 >> v1:172.21.15.90:6789/0 conn(0x7fa240147060 legacy=0x7fa24006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.632+0000 7fa23f400640 1 -- 172.21.15.82:0/2586511878 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24006d490 con 0x7fa2400cddc0 2024-03-16T17:58:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.634+0000 7fa23f400640 1 --2- 172.21.15.82:0/2586511878 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 0x7fa2400ce220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa238009e50 tx=0x7fa23802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a649be47cdcd3d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:46.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.634+0000 7fa23e000640 1 -- 172.21.15.82:0/2586511878 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23802cb00 con 0x7fa2400cddc0 2024-03-16T17:58:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.634+0000 7fa23e000640 1 -- 172.21.15.82:0/2586511878 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23802cc60 con 0x7fa2400cddc0 2024-03-16T17:58:46.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.634+0000 7fa23e000640 1 -- 172.21.15.82:0/2586511878 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2380343f0 con 0x7fa2400cddc0 2024-03-16T17:58:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.635+0000 7fa245600640 1 -- 172.21.15.82:0/2586511878 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 msgr2=0x7fa2400ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.635+0000 7fa245600640 1 --2- 172.21.15.82:0/2586511878 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 0x7fa2400ce220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa238009e50 tx=0x7fa23802bf60 comp rx=0 tx=0).stop 2024-03-16T17:58:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.635+0000 7fa245600640 1 -- 172.21.15.82:0/2586511878 shutdown_connections 2024-03-16T17:58:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.635+0000 7fa245600640 1 --2- 172.21.15.82:0/2586511878 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 0x7fa2400ce220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.635+0000 7fa245600640 1 --2- 172.21.15.82:0/2586511878 >> v2:172.21.15.90:3300/0 conn(0x7fa2401469c0 0x7fa240162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.635+0000 7fa245600640 1 -- 172.21.15.82:0/2586511878 >> 172.21.15.82:0/2586511878 conn(0x7fa2400b1640 msgr2=0x7fa2400b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.635+0000 7fa245600640 1 -- 172.21.15.82:0/2586511878 shutdown_connections 2024-03-16T17:58:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.635+0000 7fa245600640 1 -- 172.21.15.82:0/2586511878 wait complete. 2024-03-16T17:58:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.637+0000 7fa245600640 1 Processor -- start 2024-03-16T17:58:46.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.637+0000 7fa245600640 1 -- start start 2024-03-16T17:58:46.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.638+0000 7fa245600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 0x7fa2400b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.638+0000 7fa245600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2401469c0 0x7fa2401e9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.638+0000 7fa245600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa240147060 0x7fa2401e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.638+0000 7fa245600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa240135510 con 0x7fa2401469c0 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.638+0000 7fa245600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2401e84f0 con 0x7fa240147060 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.638+0000 7fa245600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2401e87d0 con 0x7fa2400cddc0 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa237e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2401469c0 0x7fa2401e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa237e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2401469c0 0x7fa2401e9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41936/0 (socket says 172.21.15.82:41936) 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa237e00640 1 -- 172.21.15.82:0/770937969 learned_addr learned my addr 172.21.15.82:0/770937969 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa23f400640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa240147060 0x7fa2401e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa23ea00640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 0x7fa2400b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa237e00640 1 -- 172.21.15.82:0/770937969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 msgr2=0x7fa2400b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa237e00640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 0x7fa2400b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa237e00640 1 -- 172.21.15.82:0/770937969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa240147060 msgr2=0x7fa2401e9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa237e00640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa240147060 0x7fa2401e9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa237e00640 1 -- 172.21.15.82:0/770937969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa238009b30 con 0x7fa2401469c0 2024-03-16T17:58:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.639+0000 7fa23f400640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa240147060 0x7fa2401e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T17:58:46.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.640+0000 7fa237e00640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2401469c0 0x7fa2401e9880 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa22800c3c0 tx=0x7fa22800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.640+0000 7fa236a00640 1 -- 172.21.15.82:0/770937969 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2280052d0 con 0x7fa2401469c0 2024-03-16T17:58:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.640+0000 7fa236a00640 1 -- 172.21.15.82:0/770937969 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228005450 con 0x7fa2401469c0 2024-03-16T17:58:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.640+0000 7fa245600640 1 -- 172.21.15.82:0/770937969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2401e8ab0 con 0x7fa2401469c0 2024-03-16T17:58:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.640+0000 7fa236a00640 1 -- 172.21.15.82:0/770937969 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228007260 con 0x7fa2401469c0 2024-03-16T17:58:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.640+0000 7fa245600640 1 -- 172.21.15.82:0/770937969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2400b5370 con 0x7fa2401469c0 2024-03-16T17:58:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.641+0000 7fa245600640 1 -- 172.21.15.82:0/770937969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24006d140 con 0x7fa2401469c0 2024-03-16T17:58:46.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.642+0000 7fa236a00640 1 -- 172.21.15.82:0/770937969 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa2280073c0 con 0x7fa2401469c0 2024-03-16T17:58:46.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.643+0000 7fa236a00640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa21c06b3b0 0x7fa21c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:46.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.643+0000 7fa236a00640 1 -- 172.21.15.82:0/770937969 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa228088d20 con 0x7fa2401469c0 2024-03-16T17:58:46.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.643+0000 7fa23ea00640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa21c06b3b0 0x7fa21c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:46.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.644+0000 7fa23ea00640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa21c06b3b0 0x7fa21c06d870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa230009ad0 tx=0x7fa230008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:46.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.648+0000 7fa236a00640 1 -- 172.21.15.82:0/770937969 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa228053c40 con 0x7fa2401469c0 2024-03-16T17:58:46.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.841+0000 7fa245600640 1 -- 172.21.15.82:0/770937969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2401620a0 con 0x7fa2401469c0 2024-03-16T17:58:46.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.842+0000 7fa236a00640 1 -- 172.21.15.82:0/770937969 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa228076070 con 0x7fa2401469c0 2024-03-16T17:58:46.845 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T17:58:46.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.844+0000 7fa245600640 1 -- 172.21.15.82:0/770937969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa21c06b3b0 msgr2=0x7fa21c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.844+0000 7fa245600640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa21c06b3b0 0x7fa21c06d870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa230009ad0 tx=0x7fa230008040 comp rx=0 tx=0).stop 2024-03-16T17:58:46.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.844+0000 7fa245600640 1 -- 172.21.15.82:0/770937969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2401469c0 msgr2=0x7fa2401e9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:46.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.844+0000 7fa245600640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2401469c0 0x7fa2401e9880 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa22800c3c0 tx=0x7fa22800c880 comp rx=0 tx=0).stop 2024-03-16T17:58:46.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.844+0000 7fa245600640 1 -- 172.21.15.82:0/770937969 shutdown_connections 2024-03-16T17:58:46.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.844+0000 7fa245600640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa21c06b3b0 0x7fa21c06d870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.844+0000 7fa245600640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa240147060 0x7fa2401e9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.844+0000 7fa245600640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2401469c0 0x7fa2401e9880 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.844+0000 7fa245600640 1 --2- 172.21.15.82:0/770937969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa2400cddc0 0x7fa2400b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:46.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.845+0000 7fa245600640 1 -- 172.21.15.82:0/770937969 >> 172.21.15.82:0/770937969 conn(0x7fa2400b1640 msgr2=0x7fa2400af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:46.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.845+0000 7fa245600640 1 -- 172.21.15.82:0/770937969 shutdown_connections 2024-03-16T17:58:46.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:46.845+0000 7fa245600640 1 -- 172.21.15.82:0/770937969 wait complete. 2024-03-16T17:58:46.866 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-16T17:58:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.058+0000 7f49c3e00640 1 Processor -- start 2024-03-16T17:58:47.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.058+0000 7f49c3e00640 1 -- start start 2024-03-16T17:58:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.060+0000 7f49c3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f49bc0cdf80 0x7f49bc0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.060+0000 7f49c3e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc0d02b0 con 0x7f49bc0cfe70 2024-03-16T17:58:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.060+0000 7f49c3e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc0b4b90 con 0x7f49bc1454d0 2024-03-16T17:58:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.060+0000 7f49c3e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49bc0cf500 con 0x7f49bc0cdf80 2024-03-16T17:58:47.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.060+0000 7f49c1e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f49bc0cfe70 0x7f49bc070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60244/0 (socket says 172.21.15.82:60244) 2024-03-16T17:58:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.060+0000 7f49c1e00640 1 -- 172.21.15.82:0/3996342650 learned_addr learned my addr 172.21.15.82:0/3996342650 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.060+0000 7f49c0a00640 1 --2- 172.21.15.82:0/3996342650 >> v2:172.21.15.90:3300/0 conn(0x7f49bc0cdf80 0x7f49bc0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.060+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251617837 0 0) 0x7f49bc0d02b0 con 0x7f49bc0cfe70 2024-03-16T17:58:47.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49a4003520 con 0x7f49bc0cfe70 2024-03-16T17:58:47.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3771133813 0 0) 0x7f49a4003520 con 0x7f49bc0cfe70 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 >> v2:172.21.15.90:3300/0 conn(0x7f49bc0cdf80 msgr2=0x7f49bc0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 --2- 172.21.15.82:0/3996342650 >> v2:172.21.15.90:3300/0 conn(0x7f49bc0cdf80 0x7f49bc0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 >> v1:172.21.15.90:6789/0 conn(0x7f49bc1454d0 legacy=0x7f49bc0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc1458b0 con 0x7f49bc0cfe70 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f49b40019f0 con 0x7f49bc0cfe70 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49c0a00640 1 --2- 172.21.15.82:0/3996342650 >> v2:172.21.15.90:3300/0 conn(0x7f49bc0cdf80 0x7f49bc0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 >> v1:172.21.15.82:6789/0 conn(0x7f49bc0cfe70 legacy=0x7f49bc070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 --2- 172.21.15.82:0/3996342650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49a40039c0 0x7f49a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 --2- 172.21.15.82:0/3996342650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49a40047c0 0x7f49a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 --2- 172.21.15.82:0/3996342650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0cf500 con 0x7f49a4005180 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a400c080 con 0x7f49a40039c0 2024-03-16T17:58:47.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.061+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a400c240 con 0x7f49a40047c0 2024-03-16T17:58:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.062+0000 7f49c1e00640 1 --2- 172.21.15.82:0/3996342650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.062+0000 7f49c1400640 1 --2- 172.21.15.82:0/3996342650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49a40039c0 0x7f49a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.062+0000 7f49c1e00640 1 -- 172.21.15.82:0/3996342650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49a40047c0 msgr2=0x7f49a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.062+0000 7f49c1e00640 1 --2- 172.21.15.82:0/3996342650 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49a40047c0 0x7f49a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.062+0000 7f49c1e00640 1 -- 172.21.15.82:0/3996342650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49a40039c0 msgr2=0x7f49a4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.062+0000 7f49c1e00640 1 --2- 172.21.15.82:0/3996342650 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49a40039c0 0x7f49a4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.062+0000 7f49c1e00640 1 -- 172.21.15.82:0/3996342650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a4003520 con 0x7f49a4005180 2024-03-16T17:58:47.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.063+0000 7f49c1e00640 1 -- 172.21.15.82:0/3996342650 reap_dead start 2024-03-16T17:58:47.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.063+0000 7f49c1e00640 1 --2- 172.21.15.82:0/3996342650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49a400b980 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f49b400a760 tx=0x7f49a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d906be4b2fc494 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:47.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.063+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4034630 con 0x7f49a4005180 2024-03-16T17:58:47.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.063+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b402f070 con 0x7f49a4005180 2024-03-16T17:58:47.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.063+0000 7f49bbe00640 1 -- 172.21.15.82:0/3996342650 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4034a10 con 0x7f49a4005180 2024-03-16T17:58:47.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.064+0000 7f49c3e00640 1 -- 172.21.15.82:0/3996342650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 msgr2=0x7f49a400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.064+0000 7f49c3e00640 1 --2- 172.21.15.82:0/3996342650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49a400b980 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f49b400a760 tx=0x7f49a40057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.064+0000 7f49c3e00640 1 -- 172.21.15.82:0/3996342650 shutdown_connections 2024-03-16T17:58:47.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.064+0000 7f49c3e00640 1 --2- 172.21.15.82:0/3996342650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49a400b980 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.064+0000 7f49c3e00640 1 -- 172.21.15.82:0/3996342650 >> 172.21.15.82:0/3996342650 conn(0x7f49bc06ae70 msgr2=0x7f49bc06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:47.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.064+0000 7f49c3e00640 1 -- 172.21.15.82:0/3996342650 shutdown_connections 2024-03-16T17:58:47.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.064+0000 7f49c3e00640 1 -- 172.21.15.82:0/3996342650 wait complete. 2024-03-16T17:58:47.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.066+0000 7f49c3e00640 1 Processor -- start 2024-03-16T17:58:47.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.066+0000 7f49c3e00640 1 -- start start 2024-03-16T17:58:47.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.067+0000 7f49c3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49bc13b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.067+0000 7f49c3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc1f4e80 0x7f49bc0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.067+0000 7f49c3e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49bc0cc1c0 0x7f49bc0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.067+0000 7f49c3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0c7cf0 con 0x7f49a4005180 2024-03-16T17:58:47.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.067+0000 7f49c3e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc1f53d0 con 0x7f49bc1f4e80 2024-03-16T17:58:47.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.067+0000 7f49c3e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc1f56b0 con 0x7f49bc0cc1c0 2024-03-16T17:58:47.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.067+0000 7f49c1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49bc13b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.067+0000 7f49c1400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49bc13b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41954/0 (socket says 172.21.15.82:41954) 2024-03-16T17:58:47.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.067+0000 7f49c1400640 1 -- 172.21.15.82:0/4294367380 learned_addr learned my addr 172.21.15.82:0/4294367380 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49c1e00640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49bc0cc1c0 0x7f49bc0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49c1400640 1 -- 172.21.15.82:0/4294367380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49bc0cc1c0 msgr2=0x7f49bc0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49c1400640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49bc0cc1c0 0x7f49bc0cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49c0a00640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc1f4e80 0x7f49bc0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49c1400640 1 -- 172.21.15.82:0/4294367380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc1f4e80 msgr2=0x7f49bc0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49c1400640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc1f4e80 0x7f49bc0c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49c1400640 1 -- 172.21.15.82:0/4294367380 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc1f5990 con 0x7f49a4005180 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49c1400640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49bc13b670 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49b000f5f0 tx=0x7f49b000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49b9600640 1 -- 172.21.15.82:0/4294367380 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00052e0 con 0x7f49a4005180 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49b9600640 1 -- 172.21.15.82:0/4294367380 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0005440 con 0x7f49a4005180 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.068+0000 7f49b9600640 1 -- 172.21.15.82:0/4294367380 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0003810 con 0x7f49a4005180 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.069+0000 7f49c3e00640 1 -- 172.21.15.82:0/4294367380 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc1f5c70 con 0x7f49a4005180 2024-03-16T17:58:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.069+0000 7f49c3e00640 1 -- 172.21.15.82:0/4294367380 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc0ccb80 con 0x7f49a4005180 2024-03-16T17:58:47.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.070+0000 7f49c3e00640 1 -- 172.21.15.82:0/4294367380 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc067f30 con 0x7f49a4005180 2024-03-16T17:58:47.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.071+0000 7f49b9600640 1 -- 172.21.15.82:0/4294367380 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f49b0003970 con 0x7f49a4005180 2024-03-16T17:58:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.071+0000 7f49b9600640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f499c06b3b0 0x7f499c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.072+0000 7f49b9600640 1 -- 172.21.15.82:0/4294367380 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f49b00897d0 con 0x7f49a4005180 2024-03-16T17:58:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.072+0000 7f49c0a00640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f499c06b3b0 0x7f499c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.072+0000 7f49c0a00640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f499c06b3b0 0x7f499c06d870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0c76e0 tx=0x7f49a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.076+0000 7f49b9600640 1 -- 172.21.15.82:0/4294367380 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f49b0054780 con 0x7f49a4005180 2024-03-16T17:58:47.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.273+0000 7f49c3e00640 1 -- 172.21.15.82:0/4294367380 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f49bc0b4640 con 0x7f49a4005180 2024-03-16T17:58:47.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.274+0000 7f49b9600640 1 -- 172.21.15.82:0/4294367380 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f49b0057e40 con 0x7f49a4005180 2024-03-16T17:58:47.277 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:47.278 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-03-16T17:58:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-03-16T17:58:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-03-16T17:58:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-03-16T17:58:47.278 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-03-16T17:58:47.278 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-03-16T17:58:47.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.276+0000 7f49c3e00640 1 -- 172.21.15.82:0/4294367380 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f499c06b3b0 msgr2=0x7f499c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.276+0000 7f49c3e00640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f499c06b3b0 0x7f499c06d870 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0c76e0 tx=0x7f49a8008040 comp rx=0 tx=0).stop 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.276+0000 7f49c3e00640 1 -- 172.21.15.82:0/4294367380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 msgr2=0x7f49bc13b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.277+0000 7f49c3e00640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49bc13b670 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49b000f5f0 tx=0x7f49b000fab0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.277+0000 7f49c3e00640 1 -- 172.21.15.82:0/4294367380 shutdown_connections 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.277+0000 7f49c3e00640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f499c06b3b0 0x7f499c06d870 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.277+0000 7f49c3e00640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f49bc0cc1c0 0x7f49bc0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.277+0000 7f49c3e00640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49bc1f4e80 0x7f49bc0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.277+0000 7f49c3e00640 1 --2- 172.21.15.82:0/4294367380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49a4005180 0x7f49bc13b670 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.277+0000 7f49c3e00640 1 -- 172.21.15.82:0/4294367380 >> 172.21.15.82:0/4294367380 conn(0x7f49bc06ae70 msgr2=0x7f49bc13d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.277+0000 7f49c3e00640 1 -- 172.21.15.82:0/4294367380 shutdown_connections 2024-03-16T17:58:47.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.277+0000 7f49c3e00640 1 -- 172.21.15.82:0/4294367380 wait complete. 2024-03-16T17:58:47.299 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-16T17:58:47.299 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-16T17:58:47.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.504+0000 7f8e0be00640 1 Processor -- start 2024-03-16T17:58:47.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.504+0000 7f8e0be00640 1 -- start start 2024-03-16T17:58:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.506+0000 7f8e0be00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8e040ccf30 0x7f8e0415a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.506+0000 7f8e0be00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e04071f40 con 0x7f8e040b2a40 2024-03-16T17:58:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.506+0000 7f8e0be00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e040ccb50 con 0x7f8e040b4270 2024-03-16T17:58:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.506+0000 7f8e0be00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e04071330 con 0x7f8e040ccf30 2024-03-16T17:58:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.506+0000 7f8e08a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8e040b2a40 0x7f8e040cc440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60248/0 (socket says 172.21.15.82:60248) 2024-03-16T17:58:47.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.506+0000 7f8e08a00640 1 -- 172.21.15.82:0/175036834 learned_addr learned my addr 172.21.15.82:0/175036834 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e09400640 1 --2- 172.21.15.82:0/175036834 >> v2:172.21.15.90:3300/0 conn(0x7f8e040ccf30 0x7f8e0415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077724611 0 0) 0x7f8e04071f40 con 0x7f8e040b2a40 2024-03-16T17:58:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dec003520 con 0x7f8e040b2a40 2024-03-16T17:58:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 470833100 0 0) 0x7f8dec003520 con 0x7f8e040b2a40 2024-03-16T17:58:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 >> v2:172.21.15.90:3300/0 conn(0x7f8e040ccf30 msgr2=0x7f8e0415a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 --2- 172.21.15.82:0/175036834 >> v2:172.21.15.90:3300/0 conn(0x7f8e040ccf30 0x7f8e0415a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 >> v1:172.21.15.90:6789/0 conn(0x7f8e040b4270 legacy=0x7f8e040b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e04071610 con 0x7f8e040b2a40 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f8df0001670 con 0x7f8e040b2a40 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 >> v1:172.21.15.82:6789/0 conn(0x7f8e040b2a40 legacy=0x7f8e040cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec003970 0x7f8dec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8dec0047c0 0x7f8dec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dec0052a0 0x7f8dec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e04071330 con 0x7f8dec003970 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec00c170 con 0x7f8dec0052a0 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.507+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec00c330 con 0x7f8dec0047c0 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.508+0000 7f8e09400640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec003970 0x7f8dec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.508+0000 7f8e09e00640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dec0052a0 0x7f8dec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.508+0000 7f8e08a00640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8dec0047c0 0x7f8dec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.508+0000 7f8e09400640 1 -- 172.21.15.82:0/175036834 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8dec0047c0 msgr2=0x7f8dec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.508+0000 7f8e09400640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8dec0047c0 0x7f8dec004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.508+0000 7f8e09400640 1 -- 172.21.15.82:0/175036834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dec0052a0 msgr2=0x7f8dec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:47.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.508+0000 7f8e09400640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dec0052a0 0x7f8dec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.508+0000 7f8e09400640 1 -- 172.21.15.82:0/175036834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec003520 con 0x7f8dec003970 2024-03-16T17:58:47.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.508+0000 7f8e09e00640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dec0052a0 0x7f8dec00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:47.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.508+0000 7f8e09e00640 1 -- 172.21.15.82:0/175036834 reap_dead start 2024-03-16T17:58:47.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.509+0000 7f8e09400640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec003970 0x7f8dec003d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009a90 tx=0x7f8dec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9e68c093672801 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:47.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.509+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df802c680 con 0x7f8dec003970 2024-03-16T17:58:47.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.509+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df802c7e0 con 0x7f8dec003970 2024-03-16T17:58:47.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.509+0000 7f8e03e00640 1 -- 172.21.15.82:0/175036834 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df802cb70 con 0x7f8dec003970 2024-03-16T17:58:47.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.510+0000 7f8e0be00640 1 -- 172.21.15.82:0/175036834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec003970 msgr2=0x7f8dec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.510+0000 7f8e0be00640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec003970 0x7f8dec003d90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009a90 tx=0x7f8dec0057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.510+0000 7f8e0be00640 1 -- 172.21.15.82:0/175036834 shutdown_connections 2024-03-16T17:58:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.510+0000 7f8e0be00640 1 --2- 172.21.15.82:0/175036834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dec003970 0x7f8dec003d90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.510+0000 7f8e0be00640 1 -- 172.21.15.82:0/175036834 >> 172.21.15.82:0/175036834 conn(0x7f8e040b1740 msgr2=0x7f8e040b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.510+0000 7f8e0be00640 1 -- 172.21.15.82:0/175036834 shutdown_connections 2024-03-16T17:58:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.510+0000 7f8e0be00640 1 -- 172.21.15.82:0/175036834 wait complete. 2024-03-16T17:58:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.512+0000 7f8e0be00640 1 Processor -- start 2024-03-16T17:58:47.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.512+0000 7f8e0be00640 1 -- start start 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e0be00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8dec003970 0x7f8e04140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e0be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e0414f8b0 0x7f8e0414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e0be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0406d4b0 0x7f8e0406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e0be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0406de70 con 0x7f8e0406d4b0 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e0be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0415a9c0 con 0x7f8e0414f8b0 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e0be00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0415ac80 con 0x7f8dec003970 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e09e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0406d4b0 0x7f8e0406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e09e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0406d4b0 0x7f8e0406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41984/0 (socket says 172.21.15.82:41984) 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e09e00640 1 -- 172.21.15.82:0/1658059318 learned_addr learned my addr 172.21.15.82:0/1658059318 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e08a00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e0414f8b0 0x7f8e0414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e09400640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8dec003970 0x7f8e04140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e09e00640 1 -- 172.21.15.82:0/1658059318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8dec003970 msgr2=0x7f8e04140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e09e00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8dec003970 0x7f8e04140aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e09e00640 1 -- 172.21.15.82:0/1658059318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e0414f8b0 msgr2=0x7f8e0414fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e09e00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e0414f8b0 0x7f8e0414fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e09e00640 1 -- 172.21.15.82:0/1658059318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0415af60 con 0x7f8e0406d4b0 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e09400640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8dec003970 0x7f8e04140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.513+0000 7f8e08a00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e0414f8b0 0x7f8e0414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.514+0000 7f8e09e00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0406d4b0 0x7f8e0406d930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8e040cd4a0 tx=0x7f8dfc00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.514+0000 7f8e01600640 1 -- 172.21.15.82:0/1658059318 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc003d80 con 0x7f8e0406d4b0 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.514+0000 7f8e01600640 1 -- 172.21.15.82:0/1658059318 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc005070 con 0x7f8e0406d4b0 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.514+0000 7f8e01600640 1 -- 172.21.15.82:0/1658059318 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc005420 con 0x7f8e0406d4b0 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.514+0000 7f8e0be00640 1 -- 172.21.15.82:0/1658059318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0415b240 con 0x7f8e0406d4b0 2024-03-16T17:58:47.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.514+0000 7f8e0be00640 1 -- 172.21.15.82:0/1658059318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0415b800 con 0x7f8e0406d4b0 2024-03-16T17:58:47.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.515+0000 7f8e0be00640 1 -- 172.21.15.82:0/1658059318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e04067f30 con 0x7f8e0406d4b0 2024-03-16T17:58:47.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.516+0000 7f8e01600640 1 -- 172.21.15.82:0/1658059318 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8dfc005580 con 0x7f8e0406d4b0 2024-03-16T17:58:47.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.517+0000 7f8e01600640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8de406b3e0 0x7f8de406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:47.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.517+0000 7f8e01600640 1 -- 172.21.15.82:0/1658059318 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8dfc089170 con 0x7f8e0406d4b0 2024-03-16T17:58:47.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.517+0000 7f8e09400640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8de406b3e0 0x7f8de406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:47.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.518+0000 7f8e09400640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8de406b3e0 0x7f8de406d8a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8df8036000 tx=0x7f8df8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:47.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.521+0000 7f8e01600640 1 -- 172.21.15.82:0/1658059318 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8dfc054110 con 0x7f8e0406d4b0 2024-03-16T17:58:47.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:47.718+0000 7f8e0be00640 1 -- 172.21.15.82:0/1658059318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8e040b2230 con 0x7f8e0406d4b0 2024-03-16T17:58:48.314 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-03-16T17:58:48.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.398+0000 7f8e01600640 1 -- 172.21.15.82:0/1658059318 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8dfc0577d0 con 0x7f8e0406d4b0 2024-03-16T17:58:48.402 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-03-16T17:58:48.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.401+0000 7f8e0be00640 1 -- 172.21.15.82:0/1658059318 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8de406b3e0 msgr2=0x7f8de406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:48.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.401+0000 7f8e0be00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8de406b3e0 0x7f8de406d8a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8df8036000 tx=0x7f8df8030000 comp rx=0 tx=0).stop 2024-03-16T17:58:48.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.401+0000 7f8e0be00640 1 -- 172.21.15.82:0/1658059318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0406d4b0 msgr2=0x7f8e0406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:48.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.401+0000 7f8e0be00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0406d4b0 0x7f8e0406d930 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8e040cd4a0 tx=0x7f8dfc00d9d0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.402+0000 7f8e0be00640 1 -- 172.21.15.82:0/1658059318 shutdown_connections 2024-03-16T17:58:48.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.402+0000 7f8e0be00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8de406b3e0 0x7f8de406d8a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.402+0000 7f8e0be00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e0406d4b0 0x7f8e0406d930 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.402+0000 7f8e0be00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8e0414f8b0 0x7f8e0414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.402+0000 7f8e0be00640 1 --2- 172.21.15.82:0/1658059318 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8dec003970 0x7f8e04140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.402+0000 7f8e0be00640 1 -- 172.21.15.82:0/1658059318 >> 172.21.15.82:0/1658059318 conn(0x7f8e040b1740 msgr2=0x7f8e040b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:48.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.402+0000 7f8e0be00640 1 -- 172.21.15.82:0/1658059318 shutdown_connections 2024-03-16T17:58:48.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.402+0000 7f8e0be00640 1 -- 172.21.15.82:0/1658059318 wait complete. 2024-03-16T17:58:48.433 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-16T17:58:48.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.627+0000 7f9f90800640 1 Processor -- start 2024-03-16T17:58:48.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.627+0000 7f9f90800640 1 -- start start 2024-03-16T17:58:48.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.629+0000 7f9f90800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f880b25e0 0x7f9f880d1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:48.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.629+0000 7f9f90800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9f880cea40 0x7f9f880ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.629+0000 7f9f90800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f880d4a40 con 0x7f9f880d0790 2024-03-16T17:58:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.629+0000 7f9f90800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f880d4d20 con 0x7f9f880cea40 2024-03-16T17:58:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.629+0000 7f9f90800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880b11b0 con 0x7f9f880b25e0 2024-03-16T17:58:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.629+0000 7f9f8de00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9f880d0790 0x7f9f880d0b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60262/0 (socket says 172.21.15.82:60262) 2024-03-16T17:58:48.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.629+0000 7f9f8de00640 1 -- 172.21.15.82:0/3695462440 learned_addr learned my addr 172.21.15.82:0/3695462440 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8d400640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f880b25e0 0x7f9f880d1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273682307 0 0) 0x7f9f880d4a40 con 0x7f9f880d0790 2024-03-16T17:58:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8e800640 1 --2- 172.21.15.82:0/3695462440 >> v2:172.21.15.90:3300/0 conn(0x7f9f880cea40 0x7f9f880ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f70003520 con 0x7f9f880d0790 2024-03-16T17:58:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4013448273 0 0) 0x7f9f70003520 con 0x7f9f880d0790 2024-03-16T17:58:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f880b25e0 msgr2=0x7f9f880d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f880b25e0 0x7f9f880d1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 >> v2:172.21.15.90:3300/0 conn(0x7f9f880cea40 msgr2=0x7f9f880ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 --2- 172.21.15.82:0/3695462440 >> v2:172.21.15.90:3300/0 conn(0x7f9f880cea40 0x7f9f880ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f880b0af0 con 0x7f9f880d0790 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f9f7c001670 con 0x7f9f880d0790 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 >> v1:172.21.15.82:6789/0 conn(0x7f9f880d0790 legacy=0x7f9f880d0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f70003970 0x7f9f70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f880b25e0 0x7f9f700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f70004d00 0x7f9f70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880d4d20 con 0x7f9f70003970 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880b11b0 con 0x7f9f70004d00 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.630+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7000b980 con 0x7f9f880b25e0 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.631+0000 7f9f8de00640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f70003970 0x7f9f70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.631+0000 7f9f8e800640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f70004d00 0x7f9f70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.631+0000 7f9f8d400640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f880b25e0 0x7f9f700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:48.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.631+0000 7f9f8de00640 1 -- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f880b25e0 msgr2=0x7f9f700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:48.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.631+0000 7f9f8de00640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f880b25e0 0x7f9f700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.631+0000 7f9f8de00640 1 -- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f70004d00 msgr2=0x7f9f70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:48.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.631+0000 7f9f8de00640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f70004d00 0x7f9f70005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.631+0000 7f9f8de00640 1 -- 172.21.15.82:0/3695462440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f70003520 con 0x7f9f70003970 2024-03-16T17:58:48.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.631+0000 7f9f8d400640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f880b25e0 0x7f9f700047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.632+0000 7f9f8de00640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f70003970 0x7f9f70003d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00c290 tx=0x7f9f700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befeb1210a46294a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.632+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c005ad0 con 0x7f9f70003970 2024-03-16T17:58:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.632+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c005c30 con 0x7f9f70003970 2024-03-16T17:58:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.632+0000 7f9f8ca00640 1 -- 172.21.15.82:0/3695462440 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c02c8d0 con 0x7f9f70003970 2024-03-16T17:58:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.633+0000 7f9f90800640 1 -- 172.21.15.82:0/3695462440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f70003970 msgr2=0x7f9f70003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.633+0000 7f9f90800640 1 --2- 172.21.15.82:0/3695462440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f70003970 0x7f9f70003d90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00c290 tx=0x7f9f700057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.633+0000 7f9f8e800640 1 -- 172.21.15.82:0/3695462440 reap_dead start 2024-03-16T17:58:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.633+0000 7f9f90800640 1 -- 172.21.15.82:0/3695462440 shutdown_connections 2024-03-16T17:58:48.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.633+0000 7f9f90800640 1 -- 172.21.15.82:0/3695462440 >> 172.21.15.82:0/3695462440 conn(0x7f9f880d3d80 msgr2=0x7f9f880d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:48.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.633+0000 7f9f90800640 1 -- 172.21.15.82:0/3695462440 shutdown_connections 2024-03-16T17:58:48.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.633+0000 7f9f90800640 1 -- 172.21.15.82:0/3695462440 wait complete. 2024-03-16T17:58:48.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.635+0000 7f9f90800640 1 Processor -- start 2024-03-16T17:58:48.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.635+0000 7f9f90800640 1 -- start start 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.636+0000 7f9f90800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f70003970 0x7f9f8815e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.636+0000 7f9f90800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f88156110 0x7f9f8815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.636+0000 7f9f90800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f88160500 0x7f9f880b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.636+0000 7f9f90800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8815eba0 con 0x7f9f88156110 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.636+0000 7f9f90800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880cd830 con 0x7f9f88160500 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.636+0000 7f9f90800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880cd270 con 0x7f9f70003970 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f88156110 0x7f9f8815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8d400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f88156110 0x7f9f8815ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42008/0 (socket says 172.21.15.82:42008) 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8d400640 1 -- 172.21.15.82:0/3320350090 learned_addr learned my addr 172.21.15.82:0/3320350090 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8e800640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f88160500 0x7f9f880b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8d400640 1 -- 172.21.15.82:0/3320350090 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f70003970 msgr2=0x7f9f8815e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:48.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8de00640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f70003970 0x7f9f8815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:48.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8d400640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f70003970 0x7f9f8815e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8d400640 1 -- 172.21.15.82:0/3320350090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f88160500 msgr2=0x7f9f880b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:48.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8d400640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f88160500 0x7f9f880b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:48.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8d400640 1 -- 172.21.15.82:0/3320350090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f880cd550 con 0x7f9f88156110 2024-03-16T17:58:48.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.637+0000 7f9f8d400640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f88156110 0x7f9f8815ffc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9f740108b0 tx=0x7f9f74010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.638+0000 7f9f82000640 1 -- 172.21.15.82:0/3320350090 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7400eb60 con 0x7f9f88156110 2024-03-16T17:58:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.638+0000 7f9f82000640 1 -- 172.21.15.82:0/3320350090 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7400ecc0 con 0x7f9f88156110 2024-03-16T17:58:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.638+0000 7f9f82000640 1 -- 172.21.15.82:0/3320350090 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7401c400 con 0x7f9f88156110 2024-03-16T17:58:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.638+0000 7f9f90800640 1 -- 172.21.15.82:0/3320350090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f88160d20 con 0x7f9f88156110 2024-03-16T17:58:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.638+0000 7f9f90800640 1 -- 172.21.15.82:0/3320350090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f88156590 con 0x7f9f88156110 2024-03-16T17:58:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.639+0000 7f9f90800640 1 -- 172.21.15.82:0/3320350090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f88067f30 con 0x7f9f88156110 2024-03-16T17:58:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.640+0000 7f9f82000640 1 -- 172.21.15.82:0/3320350090 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f9f7401c560 con 0x7f9f88156110 2024-03-16T17:58:48.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.641+0000 7f9f82000640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9f6806b3b0 0x7f9f6806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:48.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.641+0000 7f9f82000640 1 -- 172.21.15.82:0/3320350090 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9f7408a000 con 0x7f9f88156110 2024-03-16T17:58:48.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.641+0000 7f9f8de00640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9f6806b3b0 0x7f9f6806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:48.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.642+0000 7f9f8de00640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9f6806b3b0 0x7f9f6806d870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c034000 tx=0x7f9f7c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:48.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.646+0000 7f9f82000640 1 -- 172.21.15.82:0/3320350090 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9f74054ef0 con 0x7f9f88156110 2024-03-16T17:58:48.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:48.841+0000 7f9f90800640 1 -- 172.21.15.82:0/3320350090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9f880b5650 con 0x7f9f88156110 2024-03-16T17:58:49.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.405+0000 7f9f82000640 1 -- 172.21.15.82:0/3320350090 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9f740585b0 con 0x7f9f88156110 2024-03-16T17:58:49.409 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-03-16T17:58:49.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 -- 172.21.15.82:0/3320350090 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9f6806b3b0 msgr2=0x7f9f6806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:49.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9f6806b3b0 0x7f9f6806d870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c034000 tx=0x7f9f7c007000 comp rx=0 tx=0).stop 2024-03-16T17:58:49.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 -- 172.21.15.82:0/3320350090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f88156110 msgr2=0x7f9f8815ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:49.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f88156110 0x7f9f8815ffc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9f740108b0 tx=0x7f9f74010d70 comp rx=0 tx=0).stop 2024-03-16T17:58:49.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 -- 172.21.15.82:0/3320350090 shutdown_connections 2024-03-16T17:58:49.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9f6806b3b0 0x7f9f6806d870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f88160500 0x7f9f880b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f88156110 0x7f9f8815ffc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 --2- 172.21.15.82:0/3320350090 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f70003970 0x7f9f8815e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 -- 172.21.15.82:0/3320350090 >> 172.21.15.82:0/3320350090 conn(0x7f9f880d3d80 msgr2=0x7f9f880b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:49.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 -- 172.21.15.82:0/3320350090 shutdown_connections 2024-03-16T17:58:49.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.409+0000 7f9f90800640 1 -- 172.21.15.82:0/3320350090 wait complete. 2024-03-16T17:58:49.448 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-16T17:58:49.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.625+0000 7f549aa00640 1 Processor -- start 2024-03-16T17:58:49.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.625+0000 7f549aa00640 1 -- start start 2024-03-16T17:58:49.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.627+0000 7f549aa00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f54940cca20 0x7f5494166be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:49.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.627+0000 7f549aa00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54940ce550 con 0x7f5494159960 2024-03-16T17:58:49.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.627+0000 7f549aa00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549415c370 con 0x7f549406e940 2024-03-16T17:58:49.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.627+0000 7f549aa00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5494159d40 con 0x7f54940cca20 2024-03-16T17:58:49.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.627+0000 7f5493e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5494159960 0x7f549406e230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60264/0 (socket says 172.21.15.82:60264) 2024-03-16T17:58:49.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.627+0000 7f5493e00640 1 -- 172.21.15.82:0/3767883022 learned_addr learned my addr 172.21.15.82:0/3767883022 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.627+0000 7f5498a00640 1 --2- 172.21.15.82:0/3767883022 >> v2:172.21.15.90:3300/0 conn(0x7f54940cca20 0x7f5494166be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.627+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530917736 0 0) 0x7f54940ce550 con 0x7f5494159960 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.627+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5478003520 con 0x7f5494159960 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 257171673 0 0) 0x7f5478003520 con 0x7f5494159960 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 >> v2:172.21.15.90:3300/0 conn(0x7f54940cca20 msgr2=0x7f5494166be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 --2- 172.21.15.82:0/3767883022 >> v2:172.21.15.90:3300/0 conn(0x7f54940cca20 0x7f5494166be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 >> v1:172.21.15.90:6789/0 conn(0x7f549406e940 legacy=0x7f54940cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549415c650 con 0x7f5494159960 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f54840019f0 con 0x7f5494159960 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 >> v1:172.21.15.82:6789/0 conn(0x7f5494159960 legacy=0x7f549406e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:49.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5478003970 0x7f5478003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54780047c0 0x7f5478004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f547800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5494159d40 con 0x7f5478005180 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547800c120 con 0x7f5478003970 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547800c2e0 con 0x7f54780047c0 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.628+0000 7f5498a00640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f547800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.629+0000 7f5498a00640 1 -- 172.21.15.82:0/3767883022 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54780047c0 msgr2=0x7f5478004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.629+0000 7f5493400640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54780047c0 0x7f5478004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.629+0000 7f5493e00640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5478003970 0x7f5478003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.629+0000 7f5498a00640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54780047c0 0x7f5478004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.629+0000 7f5498a00640 1 -- 172.21.15.82:0/3767883022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5478003970 msgr2=0x7f5478003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.629+0000 7f5498a00640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5478003970 0x7f5478003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.629+0000 7f5498a00640 1 -- 172.21.15.82:0/3767883022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5478003520 con 0x7f5478005180 2024-03-16T17:58:49.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.630+0000 7f5498a00640 1 -- 172.21.15.82:0/3767883022 reap_dead start 2024-03-16T17:58:49.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.630+0000 7f5498a00640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f547800b980 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f548c009ac0 tx=0x7f54780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea0d683c8444149 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.630+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c004020 con 0x7f5478005180 2024-03-16T17:58:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.630+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c004180 con 0x7f5478005180 2024-03-16T17:58:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.630+0000 7f5492a00640 1 -- 172.21.15.82:0/3767883022 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c02c710 con 0x7f5478005180 2024-03-16T17:58:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.631+0000 7f549aa00640 1 -- 172.21.15.82:0/3767883022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 msgr2=0x7f547800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.631+0000 7f549aa00640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f547800b980 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f548c009ac0 tx=0x7f54780057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.631+0000 7f549aa00640 1 -- 172.21.15.82:0/3767883022 shutdown_connections 2024-03-16T17:58:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.631+0000 7f549aa00640 1 --2- 172.21.15.82:0/3767883022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f547800b980 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.631+0000 7f549aa00640 1 -- 172.21.15.82:0/3767883022 >> 172.21.15.82:0/3767883022 conn(0x7f54940b1640 msgr2=0x7f54940b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.631+0000 7f549aa00640 1 -- 172.21.15.82:0/3767883022 shutdown_connections 2024-03-16T17:58:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.631+0000 7f549aa00640 1 -- 172.21.15.82:0/3767883022 wait complete. 2024-03-16T17:58:49.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.633+0000 7f549aa00640 1 Processor -- start 2024-03-16T17:58:49.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.633+0000 7f549aa00640 1 -- start start 2024-03-16T17:58:49.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f549aa00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f54940cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:49.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f549aa00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54941e64b0 0x7f5494134d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:49.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f549aa00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54941e6930 0x7f5494135270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:49.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f549aa00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549415cff0 con 0x7f5478005180 2024-03-16T17:58:49.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f549aa00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54941e6090 con 0x7f54941e64b0 2024-03-16T17:58:49.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f549aa00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54941e56f0 con 0x7f54941e6930 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f5493e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f54940cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f5493e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f54940cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42018/0 (socket says 172.21.15.82:42018) 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f5493e00640 1 -- 172.21.15.82:0/3852123884 learned_addr learned my addr 172.21.15.82:0/3852123884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f5498a00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54941e6930 0x7f5494135270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f5493e00640 1 -- 172.21.15.82:0/3852123884 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54941e6930 msgr2=0x7f5494135270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f5493400640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54941e64b0 0x7f5494134d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f5493e00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54941e6930 0x7f5494135270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f5493e00640 1 -- 172.21.15.82:0/3852123884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54941e64b0 msgr2=0x7f5494134d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f5493e00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54941e64b0 0x7f5494134d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.634+0000 7f5493e00640 1 -- 172.21.15.82:0/3852123884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54941e59d0 con 0x7f5478005180 2024-03-16T17:58:49.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.635+0000 7f5493e00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f54940cb820 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5484001370 tx=0x7f548400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:49.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.635+0000 7f548be00640 1 -- 172.21.15.82:0/3852123884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484004b60 con 0x7f5478005180 2024-03-16T17:58:49.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.635+0000 7f548be00640 1 -- 172.21.15.82:0/3852123884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5484004cc0 con 0x7f5478005180 2024-03-16T17:58:49.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.635+0000 7f549aa00640 1 -- 172.21.15.82:0/3852123884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54941e5cb0 con 0x7f5478005180 2024-03-16T17:58:49.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.635+0000 7f548be00640 1 -- 172.21.15.82:0/3852123884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54840072d0 con 0x7f5478005180 2024-03-16T17:58:49.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.635+0000 7f549aa00640 1 -- 172.21.15.82:0/3852123884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54940cbd60 con 0x7f5478005180 2024-03-16T17:58:49.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.636+0000 7f549aa00640 1 -- 172.21.15.82:0/3852123884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5494144bd0 con 0x7f5478005180 2024-03-16T17:58:49.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.637+0000 7f548be00640 1 -- 172.21.15.82:0/3852123884 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f5484007430 con 0x7f5478005180 2024-03-16T17:58:49.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.638+0000 7f548be00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f547006b3b0 0x7f547006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:49.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.638+0000 7f548be00640 1 -- 172.21.15.82:0/3852123884 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5484089e50 con 0x7f5478005180 2024-03-16T17:58:49.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.639+0000 7f5493400640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f547006b3b0 0x7f547006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:49.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.639+0000 7f5493400640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f547006b3b0 0x7f547006d870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f54941f71a0 tx=0x7f547c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:49.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.643+0000 7f548be00640 1 -- 172.21.15.82:0/3852123884 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5484054ba0 con 0x7f5478005180 2024-03-16T17:58:49.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:49.874+0000 7f549aa00640 1 -- 172.21.15.82:0/3852123884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5494067f30 con 0x7f5478005180 2024-03-16T17:58:50.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.423+0000 7f548be00640 1 -- 172.21.15.82:0/3852123884 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f54840190c0 con 0x7f5478005180 2024-03-16T17:58:50.427 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 4 and data pool 5 2024-03-16T17:58:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.427+0000 7f549aa00640 1 -- 172.21.15.82:0/3852123884 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f547006b3b0 msgr2=0x7f547006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.427+0000 7f549aa00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f547006b3b0 0x7f547006d870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f54941f71a0 tx=0x7f547c009040 comp rx=0 tx=0).stop 2024-03-16T17:58:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.427+0000 7f549aa00640 1 -- 172.21.15.82:0/3852123884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 msgr2=0x7f54940cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.427+0000 7f549aa00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f54940cb820 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5484001370 tx=0x7f548400b4a0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.427+0000 7f549aa00640 1 -- 172.21.15.82:0/3852123884 shutdown_connections 2024-03-16T17:58:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.427+0000 7f549aa00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f547006b3b0 0x7f547006d870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.427+0000 7f549aa00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f54941e6930 0x7f5494135270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.427+0000 7f549aa00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54941e64b0 0x7f5494134d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.427+0000 7f549aa00640 1 --2- 172.21.15.82:0/3852123884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5478005180 0x7f54940cb820 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.427+0000 7f549aa00640 1 -- 172.21.15.82:0/3852123884 >> 172.21.15.82:0/3852123884 conn(0x7f54940b1640 msgr2=0x7f54940cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:50.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.428+0000 7f549aa00640 1 -- 172.21.15.82:0/3852123884 shutdown_connections 2024-03-16T17:58:50.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.428+0000 7f549aa00640 1 -- 172.21.15.82:0/3852123884 wait complete. 2024-03-16T17:58:50.473 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:50.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.649+0000 7f0612800640 1 Processor -- start 2024-03-16T17:58:50.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.649+0000 7f0612800640 1 -- start start 2024-03-16T17:58:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.650+0000 7f0612800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f060c1454d0 0x7f060c0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.650+0000 7f0612800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f060c0cdf80 0x7f060c0cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.650+0000 7f0612800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060c0b4b90 con 0x7f060c0cfe70 2024-03-16T17:58:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.650+0000 7f0612800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060c0d0320 con 0x7f060c1454d0 2024-03-16T17:58:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.650+0000 7f0612800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0cf500 con 0x7f060c0cdf80 2024-03-16T17:58:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.651+0000 7f060b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f060c1454d0 0x7f060c0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:50.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.651+0000 7f060b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f060c1454d0 0x7f060c0b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42020/0 (socket says 172.21.15.82:42020) 2024-03-16T17:58:50.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.651+0000 7f060b400640 1 -- 172.21.15.82:0/628075667 learned_addr learned my addr 172.21.15.82:0/628075667 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:50.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.651+0000 7f060aa00640 1 --2- 172.21.15.82:0/628075667 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f060c0cdf80 0x7f060c0cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:50.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.651+0000 7f060b400640 1 -- 172.21.15.82:0/628075667 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f060c0cdf80 msgr2=0x7f060c0cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:50.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.651+0000 7f060b400640 1 --2- 172.21.15.82:0/628075667 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f060c0cdf80 0x7f060c0cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.651+0000 7f060b400640 1 -- 172.21.15.82:0/628075667 >> v1:172.21.15.90:6789/0 conn(0x7f060c0cfe70 legacy=0x7f060c15f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:50.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.651+0000 7f060b400640 1 -- 172.21.15.82:0/628075667 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c1458b0 con 0x7f060c1454d0 2024-03-16T17:58:50.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.653+0000 7f060b400640 1 --2- 172.21.15.82:0/628075667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c1454d0 0x7f060c0b34c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009870 tx=0x7f05fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7338c9c8ddaded89 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:50.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.653+0000 7f060a000640 1 -- 172.21.15.82:0/628075667 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc031640 con 0x7f060c1454d0 2024-03-16T17:58:50.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.653+0000 7f060a000640 1 -- 172.21.15.82:0/628075667 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc0317a0 con 0x7f060c1454d0 2024-03-16T17:58:50.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.653+0000 7f060a000640 1 -- 172.21.15.82:0/628075667 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc031b30 con 0x7f060c1454d0 2024-03-16T17:58:50.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.653+0000 7f0612800640 1 -- 172.21.15.82:0/628075667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c1454d0 msgr2=0x7f060c0b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:50.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.653+0000 7f0612800640 1 --2- 172.21.15.82:0/628075667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c1454d0 0x7f060c0b34c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009870 tx=0x7f05fc02b8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.654+0000 7f0612800640 1 -- 172.21.15.82:0/628075667 shutdown_connections 2024-03-16T17:58:50.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.654+0000 7f0612800640 1 --2- 172.21.15.82:0/628075667 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f060c0cdf80 0x7f060c0cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.654+0000 7f0612800640 1 --2- 172.21.15.82:0/628075667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c1454d0 0x7f060c0b34c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.654+0000 7f0612800640 1 -- 172.21.15.82:0/628075667 >> 172.21.15.82:0/628075667 conn(0x7f060c06ae70 msgr2=0x7f060c06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:50.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.654+0000 7f0612800640 1 -- 172.21.15.82:0/628075667 shutdown_connections 2024-03-16T17:58:50.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.654+0000 7f0612800640 1 -- 172.21.15.82:0/628075667 wait complete. 2024-03-16T17:58:50.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.655+0000 7f0612800640 1 Processor -- start 2024-03-16T17:58:50.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.656+0000 7f0612800640 1 -- start start 2024-03-16T17:58:50.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.656+0000 7f0612800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c0cdf80 0x7f060c1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:50.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.656+0000 7f0612800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c0cfe70 0x7f060c1594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:50.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f0612800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f060c1454d0 0x7f060c1403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:50.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f0612800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0b0c70 con 0x7f060c0cdf80 2024-03-16T17:58:50.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f0612800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0b1090 con 0x7f060c0cfe70 2024-03-16T17:58:50.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f0612800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c155b80 con 0x7f060c1454d0 2024-03-16T17:58:50.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f060b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c0cdf80 0x7f060c1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:50.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f060b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c0cdf80 0x7f060c1551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42026/0 (socket says 172.21.15.82:42026) 2024-03-16T17:58:50.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f060b400640 1 -- 172.21.15.82:0/4232239475 learned_addr learned my addr 172.21.15.82:0/4232239475 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:50.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f060b400640 1 -- 172.21.15.82:0/4232239475 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f060c1454d0 msgr2=0x7f060c1403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:50.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f060b400640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f060c1454d0 0x7f060c1403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f060b400640 1 -- 172.21.15.82:0/4232239475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c0cfe70 msgr2=0x7f060c1594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:50.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f060b400640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c0cfe70 0x7f060c1594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f060b400640 1 -- 172.21.15.82:0/4232239475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0094d0 con 0x7f060c0cdf80 2024-03-16T17:58:50.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.657+0000 7f060b400640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c0cdf80 0x7f060c1551a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009870 tx=0x7f05fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:50.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.658+0000 7f0603400640 1 -- 172.21.15.82:0/4232239475 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc03ac60 con 0x7f060c0cdf80 2024-03-16T17:58:50.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.658+0000 7f0603400640 1 -- 172.21.15.82:0/4232239475 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05fc03adc0 con 0x7f060c0cdf80 2024-03-16T17:58:50.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.658+0000 7f0603400640 1 -- 172.21.15.82:0/4232239475 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05fc044630 con 0x7f060c0cdf80 2024-03-16T17:58:50.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.658+0000 7f0612800640 1 -- 172.21.15.82:0/4232239475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c0afff0 con 0x7f060c0cdf80 2024-03-16T17:58:50.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.658+0000 7f0612800640 1 -- 172.21.15.82:0/4232239475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c0b05b0 con 0x7f060c0cdf80 2024-03-16T17:58:50.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.659+0000 7f0612800640 1 -- 172.21.15.82:0/4232239475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c067f30 con 0x7f060c0cdf80 2024-03-16T17:58:50.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.660+0000 7f0603400640 1 -- 172.21.15.82:0/4232239475 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f05fc02f050 con 0x7f060c0cdf80 2024-03-16T17:58:50.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.661+0000 7f0603400640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f05ec06b3b0 0x7f05ec06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:50.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.661+0000 7f0603400640 1 -- 172.21.15.82:0/4232239475 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05fc0abac0 con 0x7f060c0cdf80 2024-03-16T17:58:50.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.661+0000 7f060aa00640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f05ec06b3b0 0x7f05ec06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:50.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.662+0000 7f060aa00640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f05ec06b3b0 0x7f05ec06d870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f060c0b3500 tx=0x7f05f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:50.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.665+0000 7f0603400640 1 -- 172.21.15.82:0/4232239475 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f05fc076710 con 0x7f060c0cdf80 2024-03-16T17:58:50.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.849+0000 7f0612800640 1 -- 172.21.15.82:0/4232239475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f060c0af4a0 con 0x7f060c0cdf80 2024-03-16T17:58:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.852+0000 7f0603400640 1 -- 172.21.15.82:0/4232239475 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16646 (secure 0 0 0) 0x7f05fc031e60 con 0x7f060c0cdf80 2024-03-16T17:58:50.856 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:50.856 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":33,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:49.875885+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:58:47.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T17:58:48.893361+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:50.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 -- 172.21.15.82:0/4232239475 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f05ec06b3b0 msgr2=0x7f05ec06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:50.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f05ec06b3b0 0x7f05ec06d870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f060c0b3500 tx=0x7f05f8008040 comp rx=0 tx=0).stop 2024-03-16T17:58:50.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 -- 172.21.15.82:0/4232239475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c0cdf80 msgr2=0x7f060c1551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:50.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c0cdf80 0x7f060c1551a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f05fc009870 tx=0x7f05fc0046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 -- 172.21.15.82:0/4232239475 shutdown_connections 2024-03-16T17:58:50.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f05ec06b3b0 0x7f05ec06d870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f060c1454d0 0x7f060c1403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f060c0cfe70 0x7f060c1594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 --2- 172.21.15.82:0/4232239475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f060c0cdf80 0x7f060c1551a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:50.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 -- 172.21.15.82:0/4232239475 >> 172.21.15.82:0/4232239475 conn(0x7f060c06ae70 msgr2=0x7f060c144410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:50.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 -- 172.21.15.82:0/4232239475 shutdown_connections 2024-03-16T17:58:50.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:50.854+0000 7f0612800640 1 -- 172.21.15.82:0/4232239475 wait complete. 2024-03-16T17:58:50.878 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.079+0000 7f98b4c00640 1 Processor -- start 2024-03-16T17:58:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.079+0000 7f98b4c00640 1 -- start start 2024-03-16T17:58:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.081+0000 7f98b4c00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f98b01595b0 0x7f98b00ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.081+0000 7f98b4c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b00b3370 con 0x7f98b0159990 2024-03-16T17:58:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.081+0000 7f98b4c00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b006dd20 con 0x7f98b006d740 2024-03-16T17:58:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.081+0000 7f98b4c00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b015f7a0 con 0x7f98b01595b0 2024-03-16T17:58:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.081+0000 7f98a6a00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f98b0159990 0x7f98b006d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60278/0 (socket says 172.21.15.82:60278) 2024-03-16T17:58:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.081+0000 7f98a6a00640 1 -- 172.21.15.82:0/1185581652 learned_addr learned my addr 172.21.15.82:0/1185581652 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:51.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.082+0000 7f98af400640 1 --2- 172.21.15.82:0/1185581652 >> v2:172.21.15.90:3300/0 conn(0x7f98b01595b0 0x7f98b00ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.082+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834488156 0 0) 0x7f98b00b3370 con 0x7f98b0159990 2024-03-16T17:58:51.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.082+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9894003520 con 0x7f98b0159990 2024-03-16T17:58:51.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.082+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4166192961 0 0) 0x7f9894003520 con 0x7f98b0159990 2024-03-16T17:58:51.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.082+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 >> v2:172.21.15.90:3300/0 conn(0x7f98b01595b0 msgr2=0x7f98b00ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:51.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.082+0000 7f98aea00640 1 --2- 172.21.15.82:0/1185581652 >> v2:172.21.15.90:3300/0 conn(0x7f98b01595b0 0x7f98b00ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.082+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 >> v1:172.21.15.90:6789/0 conn(0x7f98b006d740 legacy=0x7f98b015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.082+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b015fa80 con 0x7f98b0159990 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.082+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f9898001670 con 0x7f98b0159990 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 >> v1:172.21.15.82:6789/0 conn(0x7f98b0159990 legacy=0x7f98b006d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98aea00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f98940039e0 0x7f9894003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98aea00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98940047c0 0x7f9894004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98aea00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894005180 0x7f989400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b015f7a0 con 0x7f98940047c0 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989400c0b0 con 0x7f9894005180 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f989400c270 con 0x7f98940039e0 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98a6a00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98940047c0 0x7f9894004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98af400640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f98940039e0 0x7f9894003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98a6a00640 1 -- 172.21.15.82:0/1185581652 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f98940039e0 msgr2=0x7f9894003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98a6a00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f98940039e0 0x7f9894003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98afe00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894005180 0x7f989400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98a6a00640 1 -- 172.21.15.82:0/1185581652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894005180 msgr2=0x7f989400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98a6a00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894005180 0x7f989400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98a6a00640 1 -- 172.21.15.82:0/1185581652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894003520 con 0x7f98940047c0 2024-03-16T17:58:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98afe00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9894005180 0x7f989400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.083+0000 7f98afe00640 1 -- 172.21.15.82:0/1185581652 reap_dead start 2024-03-16T17:58:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.084+0000 7f98a6a00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98940047c0 0x7f9894004c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f98b006cc90 tx=0x7f98940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba6d6c11e21e61c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.085+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898005ad0 con 0x7f98940047c0 2024-03-16T17:58:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.085+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9898005c30 con 0x7f98940047c0 2024-03-16T17:58:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.085+0000 7f98aea00640 1 -- 172.21.15.82:0/1185581652 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989802ca90 con 0x7f98940047c0 2024-03-16T17:58:51.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.085+0000 7f98b4c00640 1 -- 172.21.15.82:0/1185581652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98940047c0 msgr2=0x7f9894004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:51.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.085+0000 7f98b4c00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98940047c0 0x7f9894004c40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f98b006cc90 tx=0x7f98940057f0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.085+0000 7f98b4c00640 1 -- 172.21.15.82:0/1185581652 shutdown_connections 2024-03-16T17:58:51.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.085+0000 7f98b4c00640 1 --2- 172.21.15.82:0/1185581652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98940047c0 0x7f9894004c40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.085+0000 7f98b4c00640 1 -- 172.21.15.82:0/1185581652 >> 172.21.15.82:0/1185581652 conn(0x7f98b006ae70 msgr2=0x7f98b006b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:51.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.086+0000 7f98b4c00640 1 -- 172.21.15.82:0/1185581652 shutdown_connections 2024-03-16T17:58:51.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.086+0000 7f98b4c00640 1 -- 172.21.15.82:0/1185581652 wait complete. 2024-03-16T17:58:51.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.087+0000 7f98b4c00640 1 Processor -- start 2024-03-16T17:58:51.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.087+0000 7f98b4c00640 1 -- start start 2024-03-16T17:58:51.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.088+0000 7f98b4c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f98940047c0 0x7f98b00adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.088+0000 7f98b4c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98b015cd70 0x7f98b015d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.088+0000 7f98b4c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b0147de0 0x7f98b0148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.088+0000 7f98b4c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b01537d0 con 0x7f98b0147de0 2024-03-16T17:58:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.088+0000 7f98b4c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b015d6f0 con 0x7f98b015cd70 2024-03-16T17:58:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.088+0000 7f98b4c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b014b260 con 0x7f98940047c0 2024-03-16T17:58:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98afe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b0147de0 0x7f98b0148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98afe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b0147de0 0x7f98b0148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42040/0 (socket says 172.21.15.82:42040) 2024-03-16T17:58:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98afe00640 1 -- 172.21.15.82:0/2845653292 learned_addr learned my addr 172.21.15.82:0/2845653292 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98af400640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f98940047c0 0x7f98b00adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98afe00640 1 -- 172.21.15.82:0/2845653292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f98940047c0 msgr2=0x7f98b00adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98afe00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f98940047c0 0x7f98b00adac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98a6a00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98b015cd70 0x7f98b015d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98afe00640 1 -- 172.21.15.82:0/2845653292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98b015cd70 msgr2=0x7f98b015d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98afe00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98b015cd70 0x7f98b015d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98afe00640 1 -- 172.21.15.82:0/2845653292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b01606e0 con 0x7f98b0147de0 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98a6a00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98b015cd70 0x7f98b015d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.089+0000 7f98afe00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b0147de0 0x7f98b0148240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f98b0072c30 tx=0x7f98a800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.090+0000 7f98a7e00640 1 -- 172.21.15.82:0/2845653292 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8003d80 con 0x7f98b0147de0 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.090+0000 7f98a7e00640 1 -- 172.21.15.82:0/2845653292 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a8005070 con 0x7f98b0147de0 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.090+0000 7f98a7e00640 1 -- 172.21.15.82:0/2845653292 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8005420 con 0x7f98b0147de0 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.090+0000 7f98b4c00640 1 -- 172.21.15.82:0/2845653292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b014b540 con 0x7f98b0147de0 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.090+0000 7f98b4c00640 1 -- 172.21.15.82:0/2845653292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b014bb00 con 0x7f98b0147de0 2024-03-16T17:58:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.091+0000 7f98b4c00640 1 -- 172.21.15.82:0/2845653292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b014bd20 con 0x7f98b0147de0 2024-03-16T17:58:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.092+0000 7f98a7e00640 1 -- 172.21.15.82:0/2845653292 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f98a8005580 con 0x7f98b0147de0 2024-03-16T17:58:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.093+0000 7f98a7e00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f988c06b3e0 0x7f988c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.093+0000 7f98a7e00640 1 -- 172.21.15.82:0/2845653292 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98a8089590 con 0x7f98b0147de0 2024-03-16T17:58:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.093+0000 7f98af400640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f988c06b3e0 0x7f988c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.093+0000 7f98af400640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f988c06b3e0 0x7f988c06d8a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009990 tx=0x7f98a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:51.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.097+0000 7f98a7e00640 1 -- 172.21.15.82:0/2845653292 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f98a8054150 con 0x7f98b0147de0 2024-03-16T17:58:51.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.293+0000 7f98b4c00640 1 -- 172.21.15.82:0/2845653292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98b004f250 con 0x7f98b0147de0 2024-03-16T17:58:51.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.295+0000 7f98a7e00640 1 -- 172.21.15.82:0/2845653292 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16646 (secure 0 0 0) 0x7f98a8005830 con 0x7f98b0147de0 2024-03-16T17:58:51.299 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:51.299 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":33,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:49.875885+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:58:47.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T17:58:48.893361+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 -- 172.21.15.82:0/2845653292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f988c06b3e0 msgr2=0x7f988c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:51.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f988c06b3e0 0x7f988c06d8a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009990 tx=0x7f98a0008040 comp rx=0 tx=0).stop 2024-03-16T17:58:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 -- 172.21.15.82:0/2845653292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b0147de0 msgr2=0x7f98b0148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b0147de0 0x7f98b0148240 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f98b0072c30 tx=0x7f98a800d9d0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 -- 172.21.15.82:0/2845653292 shutdown_connections 2024-03-16T17:58:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f988c06b3e0 0x7f988c06d8a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b0147de0 0x7f98b0148240 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f98b015cd70 0x7f98b015d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 --2- 172.21.15.82:0/2845653292 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f98940047c0 0x7f98b00adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 -- 172.21.15.82:0/2845653292 >> 172.21.15.82:0/2845653292 conn(0x7f98b006ae70 msgr2=0x7f98b00cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 -- 172.21.15.82:0/2845653292 shutdown_connections 2024-03-16T17:58:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.298+0000 7f98b4c00640 1 -- 172.21.15.82:0/2845653292 wait complete. 2024-03-16T17:58:51.319 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-16T17:58:51.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.523+0000 7f9c0a000640 1 Processor -- start 2024-03-16T17:58:51.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.523+0000 7f9c0a000640 1 -- start start 2024-03-16T17:58:51.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.524+0000 7f9c0a000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c0414b770 0x7f9c0406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.524+0000 7f9c0a000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c040cca90 0x7f9c040cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.524+0000 7f9c0a000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c040cac70 con 0x7f9c040cf130 2024-03-16T17:58:51.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.524+0000 7f9c0a000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0414bb50 con 0x7f9c0414b770 2024-03-16T17:58:51.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.524+0000 7f9c0a000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040ca060 con 0x7f9c040cca90 2024-03-16T17:58:51.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.525+0000 7f9c03400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c0414b770 0x7f9c0406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.525+0000 7f9c03400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c0414b770 0x7f9c0406cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42046/0 (socket says 172.21.15.82:42046) 2024-03-16T17:58:51.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.525+0000 7f9c03400640 1 -- 172.21.15.82:0/2240400972 learned_addr learned my addr 172.21.15.82:0/2240400972 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:51.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.525+0000 7f9c03400640 1 -- 172.21.15.82:0/2240400972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c040cca90 msgr2=0x7f9c040cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:51.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.525+0000 7f9c03400640 1 --2- 172.21.15.82:0/2240400972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c040cca90 0x7f9c040cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.525+0000 7f9c03400640 1 -- 172.21.15.82:0/2240400972 >> v1:172.21.15.90:6789/0 conn(0x7f9c040cf130 legacy=0x7f9c040cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:51.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.525+0000 7f9c03400640 1 -- 172.21.15.82:0/2240400972 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040ca340 con 0x7f9c0414b770 2024-03-16T17:58:51.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.526+0000 7f9c03400640 1 --2- 172.21.15.82:0/2240400972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 0x7f9c0406cb60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009870 tx=0x7f9bf402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c1dbb73b7a7fb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:51.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.526+0000 7f9c02a00640 1 -- 172.21.15.82:0/2240400972 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4031640 con 0x7f9c0414b770 2024-03-16T17:58:51.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.526+0000 7f9c02a00640 1 -- 172.21.15.82:0/2240400972 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf40317a0 con 0x7f9c0414b770 2024-03-16T17:58:51.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.526+0000 7f9c02a00640 1 -- 172.21.15.82:0/2240400972 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4031b30 con 0x7f9c0414b770 2024-03-16T17:58:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.527+0000 7f9c0a000640 1 -- 172.21.15.82:0/2240400972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 msgr2=0x7f9c0406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.527+0000 7f9c0a000640 1 --2- 172.21.15.82:0/2240400972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 0x7f9c0406cb60 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009870 tx=0x7f9bf402b8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.527+0000 7f9c0a000640 1 -- 172.21.15.82:0/2240400972 shutdown_connections 2024-03-16T17:58:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.527+0000 7f9c0a000640 1 --2- 172.21.15.82:0/2240400972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c040cca90 0x7f9c040cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.527+0000 7f9c0a000640 1 --2- 172.21.15.82:0/2240400972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 0x7f9c0406cb60 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.527+0000 7f9c0a000640 1 -- 172.21.15.82:0/2240400972 >> 172.21.15.82:0/2240400972 conn(0x7f9c040b1640 msgr2=0x7f9c040b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.527+0000 7f9c0a000640 1 -- 172.21.15.82:0/2240400972 shutdown_connections 2024-03-16T17:58:51.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.527+0000 7f9c0a000640 1 -- 172.21.15.82:0/2240400972 wait complete. 2024-03-16T17:58:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.529+0000 7f9c0a000640 1 Processor -- start 2024-03-16T17:58:51.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.529+0000 7f9c0a000640 1 -- start start 2024-03-16T17:58:51.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.530+0000 7f9c0a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c040cca90 0x7f9c04161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.530+0000 7f9c0a000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c040cf130 0x7f9c04148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.530+0000 7f9c0a000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 0x7f9c041f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c0a000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04147b80 con 0x7f9c0414b770 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c0a000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c04162380 con 0x7f9c040cca90 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c0a000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c041f89b0 con 0x7f9c040cf130 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 0x7f9c041f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c03e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 0x7f9c041f7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42060/0 (socket says 172.21.15.82:42060) 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c03e00640 1 -- 172.21.15.82:0/1536849060 learned_addr learned my addr 172.21.15.82:0/1536849060 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c03400640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c040cca90 0x7f9c04161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9bfbe00640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c040cf130 0x7f9c04148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c03e00640 1 -- 172.21.15.82:0/1536849060 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c040cf130 msgr2=0x7f9c04148010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c03e00640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c040cf130 0x7f9c04148010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c03e00640 1 -- 172.21.15.82:0/1536849060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c040cca90 msgr2=0x7f9c04161e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c03e00640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c040cca90 0x7f9c04161e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c03e00640 1 -- 172.21.15.82:0/1536849060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40094d0 con 0x7f9c0414b770 2024-03-16T17:58:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.531+0000 7f9c03400640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c040cca90 0x7f9c04161e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.532+0000 7f9c03e00640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 0x7f9c041f7f60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00b970 tx=0x7f9bfc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.532+0000 7f9bfb400640 1 -- 172.21.15.82:0/1536849060 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc00cb40 con 0x7f9c0414b770 2024-03-16T17:58:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.532+0000 7f9bfb400640 1 -- 172.21.15.82:0/1536849060 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bfc00ccc0 con 0x7f9c0414b770 2024-03-16T17:58:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.532+0000 7f9bfb400640 1 -- 172.21.15.82:0/1536849060 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc014680 con 0x7f9c0414b770 2024-03-16T17:58:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.532+0000 7f9c0a000640 1 -- 172.21.15.82:0/1536849060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c041f8c90 con 0x7f9c0414b770 2024-03-16T17:58:51.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.532+0000 7f9c0a000640 1 -- 172.21.15.82:0/1536849060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c04148550 con 0x7f9c0414b770 2024-03-16T17:58:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.533+0000 7f9c0a000640 1 -- 172.21.15.82:0/1536849060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c04147670 con 0x7f9c0414b770 2024-03-16T17:58:51.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.534+0000 7f9bfb400640 1 -- 172.21.15.82:0/1536849060 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f9bfc01d3b0 con 0x7f9c0414b770 2024-03-16T17:58:51.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.535+0000 7f9bfb400640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9be006b3e0 0x7f9be006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:51.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.535+0000 7f9bfb400640 1 -- 172.21.15.82:0/1536849060 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bfc089b70 con 0x7f9c0414b770 2024-03-16T17:58:51.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.535+0000 7f9c03400640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9be006b3e0 0x7f9be006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:51.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.536+0000 7f9c03400640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9be006b3e0 0x7f9be006d8a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4036000 tx=0x7f9bf403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:51.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.539+0000 7f9bfb400640 1 -- 172.21.15.82:0/1536849060 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9c04147670 con 0x7f9c0414b770 2024-03-16T17:58:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:51.776+0000 7f9c0a000640 1 -- 172.21.15.82:0/1536849060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9c04147670 con 0x7f9c0414b770 2024-03-16T17:58:52.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.470+0000 7f9bfb400640 1 -- 172.21.15.82:0/1536849060 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9c04147670 con 0x7f9c0414b770 2024-03-16T17:58:52.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.473+0000 7f9c0a000640 1 -- 172.21.15.82:0/1536849060 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9be006b3e0 msgr2=0x7f9be006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:52.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.473+0000 7f9c0a000640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9be006b3e0 0x7f9be006d8a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4036000 tx=0x7f9bf403c040 comp rx=0 tx=0).stop 2024-03-16T17:58:52.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.473+0000 7f9c0a000640 1 -- 172.21.15.82:0/1536849060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 msgr2=0x7f9c041f7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:52.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.473+0000 7f9c0a000640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 0x7f9c041f7f60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc00b970 tx=0x7f9bfc00be30 comp rx=0 tx=0).stop 2024-03-16T17:58:52.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.473+0000 7f9c0a000640 1 -- 172.21.15.82:0/1536849060 shutdown_connections 2024-03-16T17:58:52.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.474+0000 7f9c0a000640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9be006b3e0 0x7f9be006d8a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:52.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.474+0000 7f9c0a000640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c0414b770 0x7f9c041f7f60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:52.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.474+0000 7f9c0a000640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9c040cf130 0x7f9c04148010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:52.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.474+0000 7f9c0a000640 1 --2- 172.21.15.82:0/1536849060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c040cca90 0x7f9c04161e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:52.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.474+0000 7f9c0a000640 1 -- 172.21.15.82:0/1536849060 >> 172.21.15.82:0/1536849060 conn(0x7f9c040b1640 msgr2=0x7f9c040ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:52.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.474+0000 7f9c0a000640 1 -- 172.21.15.82:0/1536849060 shutdown_connections 2024-03-16T17:58:52.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.474+0000 7f9c0a000640 1 -- 172.21.15.82:0/1536849060 wait complete. 2024-03-16T17:58:52.498 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-16T17:58:52.499 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-16T17:58:52.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.705+0000 7f1faf400640 1 Processor -- start 2024-03-16T17:58:52.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.706+0000 7f1faf400640 1 -- start start 2024-03-16T17:58:52.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.707+0000 7f1faf400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1fa815e760 0x7f1fa815ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:52.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.707+0000 7f1faf400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1fa80cc380 0x7f1fa80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:52.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.708+0000 7f1faf400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa80ce8f0 con 0x7f1fa8144330 2024-03-16T17:58:52.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.708+0000 7f1faf400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa80ccd50 con 0x7f1fa815e760 2024-03-16T17:58:52.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.708+0000 7f1faf400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa815f2d0 con 0x7f1fa80cc380 2024-03-16T17:58:52.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.708+0000 7f1f9fe00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1fa815e760 0x7f1fa815ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:52.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.708+0000 7f1fad400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1fa80cc380 0x7f1fa80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:52.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.708+0000 7f1faca00640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f1fa8144330 0x7f1fa8070fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.82:33674/0 (socket says 172.21.15.82:33674) 2024-03-16T17:58:52.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.708+0000 7f1f9fe00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f1fa815e760 0x7f1fa815ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:39998/0 (socket says 172.21.15.82:39998) 2024-03-16T17:58:52.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.708+0000 7f1faca00640 1 -- 172.21.15.82:0/454258470 learned_addr learned my addr 172.21.15.82:0/454258470 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:52.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.709+0000 7f1f9fe00640 1 -- 172.21.15.82:0/454258470 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1fa80cc380 msgr2=0x7f1fa80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:52.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.709+0000 7f1f9fe00640 1 --2- 172.21.15.82:0/454258470 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1fa80cc380 0x7f1fa80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:52.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.709+0000 7f1f9fe00640 1 -- 172.21.15.82:0/454258470 >> v1:172.21.15.90:6789/0 conn(0x7f1fa8144330 legacy=0x7f1fa8070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:52.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.709+0000 7f1f9fe00640 1 -- 172.21.15.82:0/454258470 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa815f5b0 con 0x7f1fa815e760 2024-03-16T17:58:52.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.711+0000 7f1f9fe00640 1 --2- 172.21.15.82:0/454258470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa815e760 0x7f1fa815ec10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1f980097a0 tx=0x7f1f9802dac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d088f63a5dbb978 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:52.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.711+0000 7f1fa7e00640 1 -- 172.21.15.82:0/454258470 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9802f880 con 0x7f1fa815e760 2024-03-16T17:58:52.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.711+0000 7f1fa7e00640 1 -- 172.21.15.82:0/454258470 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9802f9e0 con 0x7f1fa815e760 2024-03-16T17:58:52.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.711+0000 7f1fa7e00640 1 -- 172.21.15.82:0/454258470 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9802fd70 con 0x7f1fa815e760 2024-03-16T17:58:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.712+0000 7f1faf400640 1 -- 172.21.15.82:0/454258470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa815e760 msgr2=0x7f1fa815ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.712+0000 7f1faf400640 1 --2- 172.21.15.82:0/454258470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa815e760 0x7f1fa815ec10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1f980097a0 tx=0x7f1f9802dac0 comp rx=0 tx=0).stop 2024-03-16T17:58:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.712+0000 7f1faf400640 1 -- 172.21.15.82:0/454258470 shutdown_connections 2024-03-16T17:58:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.712+0000 7f1faf400640 1 --2- 172.21.15.82:0/454258470 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1fa80cc380 0x7f1fa80cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:52.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.712+0000 7f1faf400640 1 --2- 172.21.15.82:0/454258470 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa815e760 0x7f1fa815ec10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.712+0000 7f1faf400640 1 -- 172.21.15.82:0/454258470 >> 172.21.15.82:0/454258470 conn(0x7f1fa80b1640 msgr2=0x7f1fa80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.712+0000 7f1faf400640 1 -- 172.21.15.82:0/454258470 shutdown_connections 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.712+0000 7f1faf400640 1 -- 172.21.15.82:0/454258470 wait complete. 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.714+0000 7f1faf400640 1 Processor -- start 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.714+0000 7f1faf400640 1 -- start start 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1faf400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa80cc380 0x7f1fa81424a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1faf400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa8144330 0x7f1fa81435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1faf400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1fa815e760 0x7f1fa80cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1faf400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa815b5b0 con 0x7f1fa80cc380 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1faf400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa81429e0 con 0x7f1fa8144330 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1faf400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa81effe0 con 0x7f1fa815e760 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1faca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa80cc380 0x7f1fa81424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:52.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1faca00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa80cc380 0x7f1fa81424a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42066/0 (socket says 172.21.15.82:42066) 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1faca00640 1 -- 172.21.15.82:0/108535771 learned_addr learned my addr 172.21.15.82:0/108535771 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1fad400640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1fa815e760 0x7f1fa80cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.715+0000 7f1f9fe00640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa8144330 0x7f1fa81435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1faca00640 1 -- 172.21.15.82:0/108535771 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1fa815e760 msgr2=0x7f1fa80cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1faca00640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1fa815e760 0x7f1fa80cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1faca00640 1 -- 172.21.15.82:0/108535771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa8144330 msgr2=0x7f1fa81435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1faca00640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa8144330 0x7f1fa81435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1faca00640 1 -- 172.21.15.82:0/108535771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98009400 con 0x7f1fa80cc380 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1faca00640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa80cc380 0x7f1fa81424a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000c4a0 tx=0x7f1f9000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1fa5600640 1 -- 172.21.15.82:0/108535771 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90005ce0 con 0x7f1fa80cc380 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1fa5600640 1 -- 172.21.15.82:0/108535771 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f90005e40 con 0x7f1fa80cc380 2024-03-16T17:58:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1fa5600640 1 -- 172.21.15.82:0/108535771 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90007270 con 0x7f1fa80cc380 2024-03-16T17:58:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1faf400640 1 -- 172.21.15.82:0/108535771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa81f02c0 con 0x7f1fa80cc380 2024-03-16T17:58:52.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.716+0000 7f1faf400640 1 -- 172.21.15.82:0/108535771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa8141ca0 con 0x7f1fa80cc380 2024-03-16T17:58:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.718+0000 7f1faf400640 1 -- 172.21.15.82:0/108535771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa8067f30 con 0x7f1fa80cc380 2024-03-16T17:58:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.719+0000 7f1fa5600640 1 -- 172.21.15.82:0/108535771 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f1f90007490 con 0x7f1fa80cc380 2024-03-16T17:58:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.719+0000 7f1fa5600640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1f8406b3b0 0x7f1f8406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.719+0000 7f1fa5600640 1 -- 172.21.15.82:0/108535771 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f90089060 con 0x7f1fa80cc380 2024-03-16T17:58:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.720+0000 7f1f9fe00640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1f8406b3b0 0x7f1f8406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.720+0000 7f1f9fe00640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1f8406b3b0 0x7f1f8406d870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1f98038000 tx=0x7f1f98008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:52.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.724+0000 7f1fa5600640 1 -- 172.21.15.82:0/108535771 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1f90053d70 con 0x7f1fa80cc380 2024-03-16T17:58:52.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:52.959+0000 7f1faf400640 1 -- 172.21.15.82:0/108535771 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1fa80ca540 con 0x7f1fa80cc380 2024-03-16T17:58:53.389 INFO:tasks.ceph.mds.b.smithi090.stderr:2024-03-16T17:58:53.385+0000 7fe60a000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T17:58:53.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.473+0000 7f1fa5600640 1 -- 172.21.15.82:0/108535771 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f900170c0 con 0x7f1fa80cc380 2024-03-16T17:58:53.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.477+0000 7f1faf400640 1 -- 172.21.15.82:0/108535771 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1f8406b3b0 msgr2=0x7f1f8406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:53.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.477+0000 7f1faf400640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1f8406b3b0 0x7f1f8406d870 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1f98038000 tx=0x7f1f98008030 comp rx=0 tx=0).stop 2024-03-16T17:58:53.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.477+0000 7f1faf400640 1 -- 172.21.15.82:0/108535771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa80cc380 msgr2=0x7f1fa81424a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:53.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.477+0000 7f1faf400640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa80cc380 0x7f1fa81424a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000c4a0 tx=0x7f1f9000c960 comp rx=0 tx=0).stop 2024-03-16T17:58:53.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.477+0000 7f1faf400640 1 -- 172.21.15.82:0/108535771 shutdown_connections 2024-03-16T17:58:53.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.477+0000 7f1faf400640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f1f8406b3b0 0x7f1f8406d870 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.477+0000 7f1faf400640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1fa815e760 0x7f1fa80cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.477+0000 7f1faf400640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1fa8144330 0x7f1fa81435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.477+0000 7f1faf400640 1 --2- 172.21.15.82:0/108535771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1fa80cc380 0x7f1fa81424a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.477+0000 7f1faf400640 1 -- 172.21.15.82:0/108535771 >> 172.21.15.82:0/108535771 conn(0x7f1fa80b1640 msgr2=0x7f1fa8070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:53.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.478+0000 7f1faf400640 1 -- 172.21.15.82:0/108535771 shutdown_connections 2024-03-16T17:58:53.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.478+0000 7f1faf400640 1 -- 172.21.15.82:0/108535771 wait complete. 2024-03-16T17:58:53.501 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-16T17:58:53.501 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:58:53.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.707+0000 7f58fb400640 1 Processor -- start 2024-03-16T17:58:53.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.707+0000 7f58fb400640 1 -- start start 2024-03-16T17:58:53.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58fb400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58f40d0040 0x7f58f4166c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:53.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58fb400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f40b4b90 con 0x7f58f41454d0 2024-03-16T17:58:53.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58fb400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f40cf500 con 0x7f58f40cdf80 2024-03-16T17:58:53.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58fb400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f41458b0 con 0x7f58f40d0040 2024-03-16T17:58:53.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58f9400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58f40d0040 0x7f58f4166c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:53.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58f9400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58f40d0040 0x7f58f4166c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42076/0 (socket says 172.21.15.82:42076) 2024-03-16T17:58:53.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58f9400640 1 -- 172.21.15.82:0/689934853 learned_addr learned my addr 172.21.15.82:0/689934853 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:53.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58f8a00640 1 --1- 172.21.15.82:0/689934853 >> v1:172.21.15.82:6789/0 conn(0x7f58f41454d0 0x7f58f40b34c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60282/0 (socket says 172.21.15.82:60282) 2024-03-16T17:58:53.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58f9400640 1 -- 172.21.15.82:0/689934853 >> v1:172.21.15.90:6789/0 conn(0x7f58f40cdf80 legacy=0x7f58f40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:53.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58f9400640 1 -- 172.21.15.82:0/689934853 >> v1:172.21.15.82:6789/0 conn(0x7f58f41454d0 legacy=0x7f58f40b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:53.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.709+0000 7f58f9400640 1 -- 172.21.15.82:0/689934853 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f40ce460 con 0x7f58f40d0040 2024-03-16T17:58:53.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.710+0000 7f58f9400640 1 --2- 172.21.15.82:0/689934853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40d0040 0x7f58f4166c40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009870 tx=0x7f58ec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe0bc8fd776f339 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:53.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.711+0000 7f58f3400640 1 -- 172.21.15.82:0/689934853 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec030860 con 0x7f58f40d0040 2024-03-16T17:58:53.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.711+0000 7f58f3400640 1 -- 172.21.15.82:0/689934853 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec0309c0 con 0x7f58f40d0040 2024-03-16T17:58:53.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.711+0000 7f58f3400640 1 -- 172.21.15.82:0/689934853 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec030d50 con 0x7f58f40d0040 2024-03-16T17:58:53.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.711+0000 7f58fb400640 1 -- 172.21.15.82:0/689934853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40d0040 msgr2=0x7f58f4166c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:53.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.711+0000 7f58fb400640 1 --2- 172.21.15.82:0/689934853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40d0040 0x7f58f4166c40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009870 tx=0x7f58ec02b9a0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.711+0000 7f58fb400640 1 -- 172.21.15.82:0/689934853 shutdown_connections 2024-03-16T17:58:53.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.712+0000 7f58fb400640 1 --2- 172.21.15.82:0/689934853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40d0040 0x7f58f4166c40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.712+0000 7f58fb400640 1 -- 172.21.15.82:0/689934853 >> 172.21.15.82:0/689934853 conn(0x7f58f406ae70 msgr2=0x7f58f406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:53.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.712+0000 7f58fb400640 1 -- 172.21.15.82:0/689934853 shutdown_connections 2024-03-16T17:58:53.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.712+0000 7f58fb400640 1 -- 172.21.15.82:0/689934853 wait complete. 2024-03-16T17:58:53.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.713+0000 7f58fb400640 1 Processor -- start 2024-03-16T17:58:53.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.713+0000 7f58fb400640 1 -- start start 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.714+0000 7f58fb400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40cdf80 0x7f58f413f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.714+0000 7f58fb400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58f40d0040 0x7f58f413fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.714+0000 7f58fb400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58f41454d0 0x7f58f41f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.714+0000 7f58fb400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41f49c0 con 0x7f58f40cdf80 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.714+0000 7f58fb400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41f46e0 con 0x7f58f41454d0 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.714+0000 7f58fb400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40c8f60 con 0x7f58f40d0040 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.715+0000 7f58f8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40cdf80 0x7f58f413f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.715+0000 7f58f8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40cdf80 0x7f58f413f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42090/0 (socket says 172.21.15.82:42090) 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.715+0000 7f58f8a00640 1 -- 172.21.15.82:0/2922759508 learned_addr learned my addr 172.21.15.82:0/2922759508 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.715+0000 7f58f8a00640 1 -- 172.21.15.82:0/2922759508 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58f40d0040 msgr2=0x7f58f413fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.715+0000 7f58f8a00640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58f40d0040 0x7f58f413fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.715+0000 7f58f8a00640 1 -- 172.21.15.82:0/2922759508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58f41454d0 msgr2=0x7f58f41f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:53.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.715+0000 7f58f8a00640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58f41454d0 0x7f58f41f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.715+0000 7f58f8a00640 1 -- 172.21.15.82:0/2922759508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0094d0 con 0x7f58f40cdf80 2024-03-16T17:58:53.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.715+0000 7f58f8a00640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40cdf80 0x7f58f413f540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f58e400c730 tx=0x7f58e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:53.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.716+0000 7f58f0c00640 1 -- 172.21.15.82:0/2922759508 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e400ce40 con 0x7f58f40cdf80 2024-03-16T17:58:53.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.716+0000 7f58f0c00640 1 -- 172.21.15.82:0/2922759508 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e400a640 con 0x7f58f40cdf80 2024-03-16T17:58:53.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.716+0000 7f58fb400640 1 -- 172.21.15.82:0/2922759508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f40c9240 con 0x7f58f40cdf80 2024-03-16T17:58:53.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.716+0000 7f58f0c00640 1 -- 172.21.15.82:0/2922759508 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e400a9d0 con 0x7f58f40cdf80 2024-03-16T17:58:53.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.716+0000 7f58fb400640 1 -- 172.21.15.82:0/2922759508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f40c9a40 con 0x7f58f40cdf80 2024-03-16T17:58:53.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.717+0000 7f58fb400640 1 -- 172.21.15.82:0/2922759508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f40c9760 con 0x7f58f40cdf80 2024-03-16T17:58:53.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.718+0000 7f58f0c00640 1 -- 172.21.15.82:0/2922759508 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f58e400ab30 con 0x7f58f40cdf80 2024-03-16T17:58:53.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.718+0000 7f58f0c00640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f58d406b3b0 0x7f58d406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:53.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.719+0000 7f58f0c00640 1 -- 172.21.15.82:0/2922759508 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58e4089370 con 0x7f58f40cdf80 2024-03-16T17:58:53.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.719+0000 7f58f3e00640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f58d406b3b0 0x7f58d406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:53.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.719+0000 7f58f3e00640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f58d406b3b0 0x7f58d406d870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f58f415fdf0 tx=0x7f58e00079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:53.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.723+0000 7f58f0c00640 1 -- 172.21.15.82:0/2922759508 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f58f40c9760 con 0x7f58f40cdf80 2024-03-16T17:58:53.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.959+0000 7f58fb400640 1 -- 172.21.15.82:0/2922759508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58f40c9760 con 0x7f58f40cdf80 2024-03-16T17:58:53.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.960+0000 7f58f0c00640 1 -- 172.21.15.82:0/2922759508 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4974 (secure 0 0 0) 0x7f58f40c9760 con 0x7f58f40cdf80 2024-03-16T17:58:53.964 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:53.964 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5100,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3801030113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3801030113},{"type":"v1","addr":"172.21.15.82:6836","nonce":3801030113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2266371611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2266371611},{"type":"v1","addr":"172.21.15.82:6837","nonce":2266371611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/693422260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":693422260},{"type":"v1","addr":"172.21.15.90:6834","nonce":693422260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:58:49.876364+0000","modified":"2024-03-16T17:58:52.961483+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6835/2421794786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":2421794786},{"type":"v1","addr":"172.21.15.90:6835","nonce":2421794786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5118,"qdb_cluster":[5118]},"id":2}]} 2024-03-16T17:58:53.964 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 16 2024-03-16T17:58:53.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 -- 172.21.15.82:0/2922759508 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f58d406b3b0 msgr2=0x7f58d406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:53.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f58d406b3b0 0x7f58d406d870 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f58f415fdf0 tx=0x7f58e00079b0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 -- 172.21.15.82:0/2922759508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40cdf80 msgr2=0x7f58f413f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:53.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40cdf80 0x7f58f413f540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f58e400c730 tx=0x7f58e400cbf0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 -- 172.21.15.82:0/2922759508 shutdown_connections 2024-03-16T17:58:53.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f58d406b3b0 0x7f58d406d870 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58f41454d0 0x7f58f41f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f58f40d0040 0x7f58f413fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 --2- 172.21.15.82:0/2922759508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58f40cdf80 0x7f58f413f540 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:53.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 -- 172.21.15.82:0/2922759508 >> 172.21.15.82:0/2922759508 conn(0x7f58f406ae70 msgr2=0x7f58f406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:53.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.963+0000 7f58fb400640 1 -- 172.21.15.82:0/2922759508 shutdown_connections 2024-03-16T17:58:53.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:53.964+0000 7f58fb400640 1 -- 172.21.15.82:0/2922759508 wait complete. 2024-03-16T17:58:53.987 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T17:58:54.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.186+0000 7f5b2be00640 1 Processor -- start 2024-03-16T17:58:54.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.186+0000 7f5b2be00640 1 -- start start 2024-03-16T17:58:54.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.188+0000 7f5b2be00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b241454d0 0x7f5b240b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.188+0000 7f5b2be00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f5b240cdf80 0x7f5b240cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.188+0000 7f5b2be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5b240cfe70 0x7f5b2415f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.188+0000 7f5b2be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b240d0320 con 0x7f5b240cfe70 2024-03-16T17:58:54.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.188+0000 7f5b2be00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b240b4b90 con 0x7f5b240cdf80 2024-03-16T17:58:54.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.188+0000 7f5b2be00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240cf500 con 0x7f5b241454d0 2024-03-16T17:58:54.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.188+0000 7f5b29e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5b240cfe70 0x7f5b2415f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.188+0000 7f5b29e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5b240cfe70 0x7f5b2415f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42094/0 (socket says 172.21.15.82:42094) 2024-03-16T17:58:54.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.188+0000 7f5b29e00640 1 -- 172.21.15.82:0/1023883199 learned_addr learned my addr 172.21.15.82:0/1023883199 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:54.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.188+0000 7f5b29400640 1 --2- 172.21.15.82:0/1023883199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b241454d0 0x7f5b240b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.189+0000 7f5b28a00640 1 --2- 172.21.15.82:0/1023883199 >> v2:172.21.15.90:3300/0 conn(0x7f5b240cdf80 0x7f5b240cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.189+0000 7f5b29e00640 1 -- 172.21.15.82:0/1023883199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b241454d0 msgr2=0x7f5b240b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:54.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.189+0000 7f5b29e00640 1 --2- 172.21.15.82:0/1023883199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b241454d0 0x7f5b240b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.189+0000 7f5b29e00640 1 -- 172.21.15.82:0/1023883199 >> v2:172.21.15.90:3300/0 conn(0x7f5b240cdf80 msgr2=0x7f5b240cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:54.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.189+0000 7f5b29e00640 1 --2- 172.21.15.82:0/1023883199 >> v2:172.21.15.90:3300/0 conn(0x7f5b240cdf80 0x7f5b240cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.189+0000 7f5b29e00640 1 -- 172.21.15.82:0/1023883199 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b241458b0 con 0x7f5b240cfe70 2024-03-16T17:58:54.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.190+0000 7f5b29e00640 1 --2- 172.21.15.82:0/1023883199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cfe70 0x7f5b2415f8b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b2406ede0 tx=0x7f5b1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90f83064e8aac56 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.190+0000 7f5b23e00640 1 -- 172.21.15.82:0/1023883199 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c031870 con 0x7f5b240cfe70 2024-03-16T17:58:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.190+0000 7f5b23e00640 1 -- 172.21.15.82:0/1023883199 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c0319d0 con 0x7f5b240cfe70 2024-03-16T17:58:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.190+0000 7f5b23e00640 1 -- 172.21.15.82:0/1023883199 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c031d60 con 0x7f5b240cfe70 2024-03-16T17:58:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.191+0000 7f5b2be00640 1 -- 172.21.15.82:0/1023883199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cfe70 msgr2=0x7f5b2415f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.191+0000 7f5b2be00640 1 --2- 172.21.15.82:0/1023883199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cfe70 0x7f5b2415f8b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5b2406ede0 tx=0x7f5b1c02b910 comp rx=0 tx=0).stop 2024-03-16T17:58:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.191+0000 7f5b2be00640 1 -- 172.21.15.82:0/1023883199 shutdown_connections 2024-03-16T17:58:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.191+0000 7f5b2be00640 1 --2- 172.21.15.82:0/1023883199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cfe70 0x7f5b2415f8b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.191+0000 7f5b2be00640 1 --2- 172.21.15.82:0/1023883199 >> v2:172.21.15.90:3300/0 conn(0x7f5b240cdf80 0x7f5b240cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.191+0000 7f5b2be00640 1 --2- 172.21.15.82:0/1023883199 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b241454d0 0x7f5b240b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.191+0000 7f5b2be00640 1 -- 172.21.15.82:0/1023883199 >> 172.21.15.82:0/1023883199 conn(0x7f5b2406ae70 msgr2=0x7f5b2406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:54.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.191+0000 7f5b2be00640 1 -- 172.21.15.82:0/1023883199 shutdown_connections 2024-03-16T17:58:54.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.191+0000 7f5b2be00640 1 -- 172.21.15.82:0/1023883199 wait complete. 2024-03-16T17:58:54.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.193+0000 7f5b2be00640 1 Processor -- start 2024-03-16T17:58:54.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.193+0000 7f5b2be00640 1 -- start start 2024-03-16T17:58:54.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.193+0000 7f5b2be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cdf80 0x7f5b24159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.193+0000 7f5b2be00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b240cfe70 0x7f5b2413ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b2be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b241454d0 0x7f5b241f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b2be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241f15e0 con 0x7f5b240cdf80 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b2be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2415b450 con 0x7f5b241454d0 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b2be00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241f5c10 con 0x7f5b240cfe70 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b29400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cdf80 0x7f5b24159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b29400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cdf80 0x7f5b24159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42108/0 (socket says 172.21.15.82:42108) 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b29400640 1 -- 172.21.15.82:0/2584840935 learned_addr learned my addr 172.21.15.82:0/2584840935 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b28a00640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b240cfe70 0x7f5b2413ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b29400640 1 -- 172.21.15.82:0/2584840935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b240cfe70 msgr2=0x7f5b2413ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b29400640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b240cfe70 0x7f5b2413ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b29400640 1 -- 172.21.15.82:0/2584840935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b241454d0 msgr2=0x7f5b241f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b29400640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b241454d0 0x7f5b241f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b29400640 1 -- 172.21.15.82:0/2584840935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c0094d0 con 0x7f5b240cdf80 2024-03-16T17:58:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.194+0000 7f5b28a00640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b240cfe70 0x7f5b2413ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.195+0000 7f5b29400640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cdf80 0x7f5b24159170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800d620 tx=0x7f5b1800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.195+0000 7f5b21600640 1 -- 172.21.15.82:0/2584840935 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800b620 con 0x7f5b240cdf80 2024-03-16T17:58:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.195+0000 7f5b21600640 1 -- 172.21.15.82:0/2584840935 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1800b780 con 0x7f5b240cdf80 2024-03-16T17:58:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.195+0000 7f5b2be00640 1 -- 172.21.15.82:0/2584840935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b24159750 con 0x7f5b240cdf80 2024-03-16T17:58:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.195+0000 7f5b21600640 1 -- 172.21.15.82:0/2584840935 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800bb30 con 0x7f5b240cdf80 2024-03-16T17:58:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.195+0000 7f5b2be00640 1 -- 172.21.15.82:0/2584840935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b24159d10 con 0x7f5b240cdf80 2024-03-16T17:58:54.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.196+0000 7f5b2be00640 1 -- 172.21.15.82:0/2584840935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b240b4200 con 0x7f5b240cdf80 2024-03-16T17:58:54.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.197+0000 7f5b21600640 1 -- 172.21.15.82:0/2584840935 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f5b1800bc90 con 0x7f5b240cdf80 2024-03-16T17:58:54.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.198+0000 7f5b21600640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5b0806b3b0 0x7f5b0806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.198+0000 7f5b21600640 1 -- 172.21.15.82:0/2584840935 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b1808a0c0 con 0x7f5b240cdf80 2024-03-16T17:58:54.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.198+0000 7f5b28a00640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5b0806b3b0 0x7f5b0806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.199+0000 7f5b28a00640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5b0806b3b0 0x7f5b0806d870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5b24158a40 tx=0x7f5b14008460 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:54.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.202+0000 7f5b21600640 1 -- 172.21.15.82:0/2584840935 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5b18054d10 con 0x7f5b240cdf80 2024-03-16T17:58:54.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.399+0000 7f5b2be00640 1 -- 172.21.15.82:0/2584840935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b24067f30 con 0x7f5b240cdf80 2024-03-16T17:58:54.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.401+0000 7f5b21600640 1 -- 172.21.15.82:0/2584840935 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16633 (secure 0 0 0) 0x7f5b180583d0 con 0x7f5b240cdf80 2024-03-16T17:58:54.405 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:54.405 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:51.442896+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:58:47.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T17:58:48.893361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T17:58:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.403+0000 7f5b2be00640 1 -- 172.21.15.82:0/2584840935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5b0806b3b0 msgr2=0x7f5b0806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.403+0000 7f5b2be00640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5b0806b3b0 0x7f5b0806d870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5b24158a40 tx=0x7f5b14008460 comp rx=0 tx=0).stop 2024-03-16T17:58:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.403+0000 7f5b2be00640 1 -- 172.21.15.82:0/2584840935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cdf80 msgr2=0x7f5b24159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.403+0000 7f5b2be00640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cdf80 0x7f5b24159170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800d620 tx=0x7f5b1800dae0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.404+0000 7f5b2be00640 1 -- 172.21.15.82:0/2584840935 shutdown_connections 2024-03-16T17:58:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.404+0000 7f5b2be00640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5b0806b3b0 0x7f5b0806d870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.404+0000 7f5b2be00640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b241454d0 0x7f5b241f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.404+0000 7f5b2be00640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5b240cfe70 0x7f5b2413ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.404+0000 7f5b2be00640 1 --2- 172.21.15.82:0/2584840935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b240cdf80 0x7f5b24159170 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.404+0000 7f5b2be00640 1 -- 172.21.15.82:0/2584840935 >> 172.21.15.82:0/2584840935 conn(0x7f5b2406ae70 msgr2=0x7f5b240b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:54.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.404+0000 7f5b2be00640 1 -- 172.21.15.82:0/2584840935 shutdown_connections 2024-03-16T17:58:54.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.404+0000 7f5b2be00640 1 -- 172.21.15.82:0/2584840935 wait complete. 2024-03-16T17:58:54.428 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-16T17:58:54.428 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T17:58:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.641+0000 7f0985400640 1 Processor -- start 2024-03-16T17:58:54.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.641+0000 7f0985400640 1 -- start start 2024-03-16T17:58:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.642+0000 7f0985400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f09801440a0 0x7f098006db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.642+0000 7f0985400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0980144480 0x7f098006e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.642+0000 7f0985400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f098015e430 0x7f098015e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.642+0000 7f0985400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098006e5c0 con 0x7f09801440a0 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.642+0000 7f0985400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09800ccff0 con 0x7f0980144480 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.642+0000 7f0985400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098015f1a0 con 0x7f098015e430 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.643+0000 7f097fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f09801440a0 0x7f098006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.643+0000 7f097fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f09801440a0 0x7f098006db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42116/0 (socket says 172.21.15.82:42116) 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.643+0000 7f097fe00640 1 -- 172.21.15.82:0/1952829012 learned_addr learned my addr 172.21.15.82:0/1952829012 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.643+0000 7f097fe00640 1 -- 172.21.15.82:0/1952829012 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f098015e430 msgr2=0x7f098015e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.643+0000 7f097fe00640 1 --2- 172.21.15.82:0/1952829012 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f098015e430 0x7f098015e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.643+0000 7f097fe00640 1 -- 172.21.15.82:0/1952829012 >> v2:172.21.15.90:3300/0 conn(0x7f0980144480 msgr2=0x7f098006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.643+0000 7f097fe00640 1 --2- 172.21.15.82:0/1952829012 >> v2:172.21.15.90:3300/0 conn(0x7f0980144480 0x7f098006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.643+0000 7f097fe00640 1 -- 172.21.15.82:0/1952829012 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800cc230 con 0x7f09801440a0 2024-03-16T17:58:54.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.644+0000 7f097fe00640 1 --2- 172.21.15.82:0/1952829012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09801440a0 0x7f098006db40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f098015e350 tx=0x7f097002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa59eb0c615ca76 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.644+0000 7f097ea00640 1 -- 172.21.15.82:0/1952829012 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970031860 con 0x7f09801440a0 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f097ea00640 1 -- 172.21.15.82:0/1952829012 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09700319c0 con 0x7f09801440a0 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f097ea00640 1 -- 172.21.15.82:0/1952829012 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970031d50 con 0x7f09801440a0 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f0985400640 1 -- 172.21.15.82:0/1952829012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09801440a0 msgr2=0x7f098006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f0985400640 1 --2- 172.21.15.82:0/1952829012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09801440a0 0x7f098006db40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f098015e350 tx=0x7f097002b900 comp rx=0 tx=0).stop 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f0985400640 1 -- 172.21.15.82:0/1952829012 shutdown_connections 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f0985400640 1 --2- 172.21.15.82:0/1952829012 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f098015e430 0x7f098015e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f0985400640 1 --2- 172.21.15.82:0/1952829012 >> v2:172.21.15.90:3300/0 conn(0x7f0980144480 0x7f098006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f0985400640 1 --2- 172.21.15.82:0/1952829012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09801440a0 0x7f098006db40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f0985400640 1 -- 172.21.15.82:0/1952829012 >> 172.21.15.82:0/1952829012 conn(0x7f098006ae70 msgr2=0x7f098006b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f0985400640 1 -- 172.21.15.82:0/1952829012 shutdown_connections 2024-03-16T17:58:54.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.645+0000 7f0985400640 1 -- 172.21.15.82:0/1952829012 wait complete. 2024-03-16T17:58:54.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.647+0000 7f0985400640 1 Processor -- start 2024-03-16T17:58:54.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.647+0000 7f0985400640 1 -- start start 2024-03-16T17:58:54.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.648+0000 7f0985400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f09801440a0 0x7f098015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.648+0000 7f0985400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980144480 0x7f09800cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.648+0000 7f0985400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f098015e430 0x7f09800cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.648+0000 7f0985400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800cd660 con 0x7f0980144480 2024-03-16T17:58:54.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.648+0000 7f0985400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800cd8b0 con 0x7f098015e430 2024-03-16T17:58:54.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.648+0000 7f0985400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09800cdb90 con 0x7f09801440a0 2024-03-16T17:58:54.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.648+0000 7f097f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980144480 0x7f09800cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.648+0000 7f097f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980144480 0x7f09800cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42120/0 (socket says 172.21.15.82:42120) 2024-03-16T17:58:54.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.648+0000 7f097f400640 1 -- 172.21.15.82:0/2431392143 learned_addr learned my addr 172.21.15.82:0/2431392143 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:54.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.649+0000 7f097fe00640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f09801440a0 0x7f098015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.649+0000 7f097f400640 1 -- 172.21.15.82:0/2431392143 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f09801440a0 msgr2=0x7f098015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:54.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.649+0000 7f097f400640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f09801440a0 0x7f098015a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.649+0000 7f097f400640 1 -- 172.21.15.82:0/2431392143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f098015e430 msgr2=0x7f09800cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:54.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.649+0000 7f0984a00640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f098015e430 0x7f09800cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.649+0000 7f097f400640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f098015e430 0x7f09800cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.649+0000 7f097f400640 1 -- 172.21.15.82:0/2431392143 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700094d0 con 0x7f0980144480 2024-03-16T17:58:54.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.649+0000 7f0984a00640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f098015e430 0x7f09800cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:54.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.649+0000 7f097f400640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980144480 0x7f09800cf490 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f096800b4a0 tx=0x7f096800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:54.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.649+0000 7f0977e00640 1 -- 172.21.15.82:0/2431392143 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968010610 con 0x7f0980144480 2024-03-16T17:58:54.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.650+0000 7f0977e00640 1 -- 172.21.15.82:0/2431392143 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0968010770 con 0x7f0980144480 2024-03-16T17:58:54.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.650+0000 7f0977e00640 1 -- 172.21.15.82:0/2431392143 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968010b20 con 0x7f0980144480 2024-03-16T17:58:54.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.650+0000 7f0985400640 1 -- 172.21.15.82:0/2431392143 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09800cde70 con 0x7f0980144480 2024-03-16T17:58:54.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.650+0000 7f0985400640 1 -- 172.21.15.82:0/2431392143 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09800ce430 con 0x7f0980144480 2024-03-16T17:58:54.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.651+0000 7f0985400640 1 -- 172.21.15.82:0/2431392143 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098004f250 con 0x7f0980144480 2024-03-16T17:58:54.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.652+0000 7f0977e00640 1 -- 172.21.15.82:0/2431392143 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f0968010c80 con 0x7f0980144480 2024-03-16T17:58:54.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.652+0000 7f0977e00640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f095c06b3b0 0x7f095c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:54.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.653+0000 7f0977e00640 1 -- 172.21.15.82:0/2431392143 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09680898d0 con 0x7f0980144480 2024-03-16T17:58:54.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.653+0000 7f097fe00640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f095c06b3b0 0x7f095c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:54.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.654+0000 7f097fe00640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f095c06b3b0 0x7f095c06d870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0970036000 tx=0x7f097003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:54.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.657+0000 7f0977e00640 1 -- 172.21.15.82:0/2431392143 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0968054520 con 0x7f0980144480 2024-03-16T17:58:54.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.849+0000 7f0985400640 1 -- 172.21.15.82:0/2431392143 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09800ccad0 con 0x7f095c06b3b0 2024-03-16T17:58:54.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.862+0000 7f0977e00640 1 -- 172.21.15.82:0/2431392143 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251950 (secure 0 0 0) 0x7f0970031cb0 con 0x7f095c06b3b0 2024-03-16T17:58:54.869 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:54.871 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T17:58:54.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.867+0000 7f0985400640 1 -- 172.21.15.82:0/2431392143 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f095c06b3b0 msgr2=0x7f095c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:54.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.867+0000 7f0985400640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f095c06b3b0 0x7f095c06d870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0970036000 tx=0x7f097003c040 comp rx=0 tx=0).stop 2024-03-16T17:58:54.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.867+0000 7f0985400640 1 -- 172.21.15.82:0/2431392143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980144480 msgr2=0x7f09800cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:54.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.867+0000 7f0985400640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980144480 0x7f09800cf490 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f096800b4a0 tx=0x7f096800b960 comp rx=0 tx=0).stop 2024-03-16T17:58:54.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.868+0000 7f0985400640 1 -- 172.21.15.82:0/2431392143 shutdown_connections 2024-03-16T17:58:54.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.868+0000 7f0985400640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f095c06b3b0 0x7f095c06d870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.868+0000 7f0985400640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f098015e430 0x7f09800cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.868+0000 7f0985400640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0980144480 0x7f09800cf490 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.868+0000 7f0985400640 1 --2- 172.21.15.82:0/2431392143 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f09801440a0 0x7f098015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.868+0000 7f0985400640 1 -- 172.21.15.82:0/2431392143 >> 172.21.15.82:0/2431392143 conn(0x7f098006ae70 msgr2=0x7f098006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.868+0000 7f0985400640 1 -- 172.21.15.82:0/2431392143 shutdown_connections 2024-03-16T17:58:54.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:54.868+0000 7f0985400640 1 -- 172.21.15.82:0/2431392143 wait complete. 2024-03-16T17:58:54.891 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-03-16T17:58:54.033327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220396,"kb_used_data":4200,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754324,"statfs":{"total":773094113280,"available":772868427776,"internally_reserved":0,"allocated":4300800,"data_stored":2385612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.632590"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.072327+0000","last_change":"2024-03-16T17:58:50.463404+0000","last_active":"2024-03-16T17:58:52.072327+0000","last_peered":"2024-03-16T17:58:52.072327+0000","last_clean":"2024-03-16T17:58:52.072327+0000","last_became_active":"2024-03-16T17:58:50.463099+0000","last_became_peered":"2024-03-16T17:58:50.463099+0000","last_unstale":"2024-03-16T17:58:52.072327+0000","last_undegraded":"2024-03-16T17:58:52.072327+0000","last_fullsized":"2024-03-16T17:58:52.072327+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:35:50.299745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566333+0000","last_change":"2024-03-16T17:58:50.452964+0000","last_active":"2024-03-16T17:58:51.566333+0000","last_peered":"2024-03-16T17:58:51.566333+0000","last_clean":"2024-03-16T17:58:51.566333+0000","last_became_active":"2024-03-16T17:58:50.452700+0000","last_became_peered":"2024-03-16T17:58:50.452700+0000","last_unstale":"2024-03-16T17:58:51.566333+0000","last_undegraded":"2024-03-16T17:58:51.566333+0000","last_fullsized":"2024-03-16T17:58:51.566333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:14:57.330119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071274+0000","last_change":"2024-03-16T17:58:50.461858+0000","last_active":"2024-03-16T17:58:52.071274+0000","last_peered":"2024-03-16T17:58:52.071274+0000","last_clean":"2024-03-16T17:58:52.071274+0000","last_became_active":"2024-03-16T17:58:50.461515+0000","last_became_peered":"2024-03-16T17:58:50.461515+0000","last_unstale":"2024-03-16T17:58:52.071274+0000","last_undegraded":"2024-03-16T17:58:52.071274+0000","last_fullsized":"2024-03-16T17:58:52.071274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:03:27.488348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070236+0000","last_change":"2024-03-16T17:58:50.464429+0000","last_active":"2024-03-16T17:58:52.070236+0000","last_peered":"2024-03-16T17:58:52.070236+0000","last_clean":"2024-03-16T17:58:52.070236+0000","last_became_active":"2024-03-16T17:58:50.464155+0000","last_became_peered":"2024-03-16T17:58:50.464155+0000","last_unstale":"2024-03-16T17:58:52.070236+0000","last_undegraded":"2024-03-16T17:58:52.070236+0000","last_fullsized":"2024-03-16T17:58:52.070236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:46:45.157408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455241+0000","last_change":"2024-03-16T17:58:50.475001+0000","last_active":"2024-03-16T17:58:51.455241+0000","last_peered":"2024-03-16T17:58:51.455241+0000","last_clean":"2024-03-16T17:58:51.455241+0000","last_became_active":"2024-03-16T17:58:50.474725+0000","last_became_peered":"2024-03-16T17:58:50.474725+0000","last_unstale":"2024-03-16T17:58:51.455241+0000","last_undegraded":"2024-03-16T17:58:51.455241+0000","last_fullsized":"2024-03-16T17:58:51.455241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:15:56.094834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.565988+0000","last_change":"2024-03-16T17:58:50.444215+0000","last_active":"2024-03-16T17:58:51.565988+0000","last_peered":"2024-03-16T17:58:51.565988+0000","last_clean":"2024-03-16T17:58:51.565988+0000","last_became_active":"2024-03-16T17:58:50.443676+0000","last_became_peered":"2024-03-16T17:58:50.443676+0000","last_unstale":"2024-03-16T17:58:51.565988+0000","last_undegraded":"2024-03-16T17:58:51.565988+0000","last_fullsized":"2024-03-16T17:58:51.565988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:11:34.158762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070019+0000","last_change":"2024-03-16T17:58:50.459374+0000","last_active":"2024-03-16T17:58:52.070019+0000","last_peered":"2024-03-16T17:58:52.070019+0000","last_clean":"2024-03-16T17:58:52.070019+0000","last_became_active":"2024-03-16T17:58:50.459010+0000","last_became_peered":"2024-03-16T17:58:50.459010+0000","last_unstale":"2024-03-16T17:58:52.070019+0000","last_undegraded":"2024-03-16T17:58:52.070019+0000","last_fullsized":"2024-03-16T17:58:52.070019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:53:55.960247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069948+0000","last_change":"2024-03-16T17:58:50.468977+0000","last_active":"2024-03-16T17:58:52.069948+0000","last_peered":"2024-03-16T17:58:52.069948+0000","last_clean":"2024-03-16T17:58:52.069948+0000","last_became_active":"2024-03-16T17:58:50.468622+0000","last_became_peered":"2024-03-16T17:58:50.468622+0000","last_unstale":"2024-03-16T17:58:52.069948+0000","last_undegraded":"2024-03-16T17:58:52.069948+0000","last_fullsized":"2024-03-16T17:58:52.069948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:04:31.126786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.457060+0000","last_change":"2024-03-16T17:58:50.475809+0000","last_active":"2024-03-16T17:58:51.457060+0000","last_peered":"2024-03-16T17:58:51.457060+0000","last_clean":"2024-03-16T17:58:51.457060+0000","last_became_active":"2024-03-16T17:58:50.475419+0000","last_became_peered":"2024-03-16T17:58:50.475419+0000","last_unstale":"2024-03-16T17:58:51.457060+0000","last_undegraded":"2024-03-16T17:58:51.457060+0000","last_fullsized":"2024-03-16T17:58:51.457060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:28:50.473379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070335+0000","last_change":"2024-03-16T17:58:50.470761+0000","last_active":"2024-03-16T17:58:52.070335+0000","last_peered":"2024-03-16T17:58:52.070335+0000","last_clean":"2024-03-16T17:58:52.070335+0000","last_became_active":"2024-03-16T17:58:50.468543+0000","last_became_peered":"2024-03-16T17:58:50.468543+0000","last_unstale":"2024-03-16T17:58:52.070335+0000","last_undegraded":"2024-03-16T17:58:52.070335+0000","last_fullsized":"2024-03-16T17:58:52.070335+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:24:56.380097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.074163+0000","last_change":"2024-03-16T17:58:50.467169+0000","last_active":"2024-03-16T17:58:52.074163+0000","last_peered":"2024-03-16T17:58:52.074163+0000","last_clean":"2024-03-16T17:58:52.074163+0000","last_became_active":"2024-03-16T17:58:50.465308+0000","last_became_peered":"2024-03-16T17:58:50.465308+0000","last_unstale":"2024-03-16T17:58:52.074163+0000","last_undegraded":"2024-03-16T17:58:52.074163+0000","last_fullsized":"2024-03-16T17:58:52.074163+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:03:27.488348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069965+0000","last_change":"2024-03-16T17:58:50.442059+0000","last_active":"2024-03-16T17:58:52.069965+0000","last_peered":"2024-03-16T17:58:52.069965+0000","last_clean":"2024-03-16T17:58:52.069965+0000","last_became_active":"2024-03-16T17:58:50.439457+0000","last_became_peered":"2024-03-16T17:58:50.439457+0000","last_unstale":"2024-03-16T17:58:52.069965+0000","last_undegraded":"2024-03-16T17:58:52.069965+0000","last_fullsized":"2024-03-16T17:58:52.069965+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:14:57.330119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455470+0000","last_change":"2024-03-16T17:58:50.476284+0000","last_active":"2024-03-16T17:58:51.455470+0000","last_peered":"2024-03-16T17:58:51.455470+0000","last_clean":"2024-03-16T17:58:51.455470+0000","last_became_active":"2024-03-16T17:58:50.475925+0000","last_became_peered":"2024-03-16T17:58:50.475925+0000","last_unstale":"2024-03-16T17:58:51.455470+0000","last_undegraded":"2024-03-16T17:58:51.455470+0000","last_fullsized":"2024-03-16T17:58:51.455470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:15:14.236101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566080+0000","last_change":"2024-03-16T17:58:50.457152+0000","last_active":"2024-03-16T17:58:51.566080+0000","last_peered":"2024-03-16T17:58:51.566080+0000","last_clean":"2024-03-16T17:58:51.566080+0000","last_became_active":"2024-03-16T17:58:50.456805+0000","last_became_peered":"2024-03-16T17:58:50.456805+0000","last_unstale":"2024-03-16T17:58:51.566080+0000","last_undegraded":"2024-03-16T17:58:51.566080+0000","last_fullsized":"2024-03-16T17:58:51.566080+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:33:16.046811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069781+0000","last_change":"2024-03-16T17:58:50.446637+0000","last_active":"2024-03-16T17:58:52.069781+0000","last_peered":"2024-03-16T17:58:52.069781+0000","last_clean":"2024-03-16T17:58:52.069781+0000","last_became_active":"2024-03-16T17:58:50.446288+0000","last_became_peered":"2024-03-16T17:58:50.446288+0000","last_unstale":"2024-03-16T17:58:52.069781+0000","last_undegraded":"2024-03-16T17:58:52.069781+0000","last_fullsized":"2024-03-16T17:58:52.069781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:18:17.056096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071357+0000","last_change":"2024-03-16T17:58:50.475384+0000","last_active":"2024-03-16T17:58:52.071357+0000","last_peered":"2024-03-16T17:58:52.071357+0000","last_clean":"2024-03-16T17:58:52.071357+0000","last_became_active":"2024-03-16T17:58:50.474977+0000","last_became_peered":"2024-03-16T17:58:50.474977+0000","last_unstale":"2024-03-16T17:58:52.071357+0000","last_undegraded":"2024-03-16T17:58:52.071357+0000","last_fullsized":"2024-03-16T17:58:52.071357+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:33:16.046811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.074586+0000","last_change":"2024-03-16T17:58:50.467327+0000","last_active":"2024-03-16T17:58:52.074586+0000","last_peered":"2024-03-16T17:58:52.074586+0000","last_clean":"2024-03-16T17:58:52.074586+0000","last_became_active":"2024-03-16T17:58:50.466879+0000","last_became_peered":"2024-03-16T17:58:50.466879+0000","last_unstale":"2024-03-16T17:58:52.074586+0000","last_undegraded":"2024-03-16T17:58:52.074586+0000","last_fullsized":"2024-03-16T17:58:52.074586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:06:02.239701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566715+0000","last_change":"2024-03-16T17:58:50.462782+0000","last_active":"2024-03-16T17:58:51.566715+0000","last_peered":"2024-03-16T17:58:51.566715+0000","last_clean":"2024-03-16T17:58:51.566715+0000","last_became_active":"2024-03-16T17:58:50.462504+0000","last_became_peered":"2024-03-16T17:58:50.462504+0000","last_unstale":"2024-03-16T17:58:51.566715+0000","last_undegraded":"2024-03-16T17:58:51.566715+0000","last_fullsized":"2024-03-16T17:58:51.566715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:06:02.239701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566298+0000","last_change":"2024-03-16T17:58:50.463054+0000","last_active":"2024-03-16T17:58:51.566298+0000","last_peered":"2024-03-16T17:58:51.566298+0000","last_clean":"2024-03-16T17:58:51.566298+0000","last_became_active":"2024-03-16T17:58:50.462563+0000","last_became_peered":"2024-03-16T17:58:50.462563+0000","last_unstale":"2024-03-16T17:58:51.566298+0000","last_undegraded":"2024-03-16T17:58:51.566298+0000","last_fullsized":"2024-03-16T17:58:51.566298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:18:17.056096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455736+0000","last_change":"2024-03-16T17:58:50.465327+0000","last_active":"2024-03-16T17:58:51.455736+0000","last_peered":"2024-03-16T17:58:51.455736+0000","last_clean":"2024-03-16T17:58:51.455736+0000","last_became_active":"2024-03-16T17:58:50.463433+0000","last_became_peered":"2024-03-16T17:58:50.463433+0000","last_unstale":"2024-03-16T17:58:51.455736+0000","last_undegraded":"2024-03-16T17:58:51.455736+0000","last_fullsized":"2024-03-16T17:58:51.455736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:33.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456464+0000","last_change":"2024-03-16T17:58:50.448625+0000","last_active":"2024-03-16T17:58:51.456464+0000","last_peered":"2024-03-16T17:58:51.456464+0000","last_clean":"2024-03-16T17:58:51.456464+0000","last_became_active":"2024-03-16T17:58:50.448235+0000","last_became_peered":"2024-03-16T17:58:50.448235+0000","last_unstale":"2024-03-16T17:58:51.456464+0000","last_undegraded":"2024-03-16T17:58:51.456464+0000","last_fullsized":"2024-03-16T17:58:51.456464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:22:41.995233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456489+0000","last_change":"2024-03-16T17:58:50.446583+0000","last_active":"2024-03-16T17:58:51.456489+0000","last_peered":"2024-03-16T17:58:51.456489+0000","last_clean":"2024-03-16T17:58:51.456489+0000","last_became_active":"2024-03-16T17:58:50.446296+0000","last_became_peered":"2024-03-16T17:58:50.446296+0000","last_unstale":"2024-03-16T17:58:51.456489+0000","last_undegraded":"2024-03-16T17:58:51.456489+0000","last_fullsized":"2024-03-16T17:58:51.456489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:21:28.840567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.453952+0000","last_change":"2024-03-16T17:58:50.456867+0000","last_active":"2024-03-16T17:58:51.453952+0000","last_peered":"2024-03-16T17:58:51.453952+0000","last_clean":"2024-03-16T17:58:51.453952+0000","last_became_active":"2024-03-16T17:58:50.456597+0000","last_became_peered":"2024-03-16T17:58:50.456597+0000","last_unstale":"2024-03-16T17:58:51.453952+0000","last_undegraded":"2024-03-16T17:58:51.453952+0000","last_fullsized":"2024-03-16T17:58:51.453952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:23:47.737703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069116+0000","last_change":"2024-03-16T17:58:50.472775+0000","last_active":"2024-03-16T17:58:52.069116+0000","last_peered":"2024-03-16T17:58:52.069116+0000","last_clean":"2024-03-16T17:58:52.069116+0000","last_became_active":"2024-03-16T17:58:50.472197+0000","last_became_peered":"2024-03-16T17:58:50.472197+0000","last_unstale":"2024-03-16T17:58:52.069116+0000","last_undegraded":"2024-03-16T17:58:52.069116+0000","last_fullsized":"2024-03-16T17:58:52.069116+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:11:34.158762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070506+0000","last_change":"2024-03-16T17:58:50.473039+0000","last_active":"2024-03-16T17:58:52.070506+0000","last_peered":"2024-03-16T17:58:52.070506+0000","last_clean":"2024-03-16T17:58:52.070506+0000","last_became_active":"2024-03-16T17:58:50.472760+0000","last_became_peered":"2024-03-16T17:58:50.472760+0000","last_unstale":"2024-03-16T17:58:52.070506+0000","last_undegraded":"2024-03-16T17:58:52.070506+0000","last_fullsized":"2024-03-16T17:58:52.070506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:36:18.198333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566803+0000","last_change":"2024-03-16T17:58:50.467268+0000","last_active":"2024-03-16T17:58:51.566803+0000","last_peered":"2024-03-16T17:58:51.566803+0000","last_clean":"2024-03-16T17:58:51.566803+0000","last_became_active":"2024-03-16T17:58:50.466889+0000","last_became_peered":"2024-03-16T17:58:50.466889+0000","last_unstale":"2024-03-16T17:58:51.566803+0000","last_undegraded":"2024-03-16T17:58:51.566803+0000","last_fullsized":"2024-03-16T17:58:51.566803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:12:09.875275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070686+0000","last_change":"2024-03-16T17:58:50.475482+0000","last_active":"2024-03-16T17:58:52.070686+0000","last_peered":"2024-03-16T17:58:52.070686+0000","last_clean":"2024-03-16T17:58:52.070686+0000","last_became_active":"2024-03-16T17:58:50.475185+0000","last_became_peered":"2024-03-16T17:58:50.475185+0000","last_unstale":"2024-03-16T17:58:52.070686+0000","last_undegraded":"2024-03-16T17:58:52.070686+0000","last_fullsized":"2024-03-16T17:58:52.070686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:48:12.585703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070224+0000","last_change":"2024-03-16T17:58:50.447944+0000","last_active":"2024-03-16T17:58:52.070224+0000","last_peered":"2024-03-16T17:58:52.070224+0000","last_clean":"2024-03-16T17:58:52.070224+0000","last_became_active":"2024-03-16T17:58:50.447395+0000","last_became_peered":"2024-03-16T17:58:50.447395+0000","last_unstale":"2024-03-16T17:58:52.070224+0000","last_undegraded":"2024-03-16T17:58:52.070224+0000","last_fullsized":"2024-03-16T17:58:52.070224+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:19:18.993584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456683+0000","last_change":"2024-03-16T17:58:50.455905+0000","last_active":"2024-03-16T17:58:51.456683+0000","last_peered":"2024-03-16T17:58:51.456683+0000","last_clean":"2024-03-16T17:58:51.456683+0000","last_became_active":"2024-03-16T17:58:50.454469+0000","last_became_peered":"2024-03-16T17:58:50.454469+0000","last_unstale":"2024-03-16T17:58:51.456683+0000","last_undegraded":"2024-03-16T17:58:51.456683+0000","last_fullsized":"2024-03-16T17:58:51.456683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:36:50.817760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071830+0000","last_change":"2024-03-16T17:58:50.458671+0000","last_active":"2024-03-16T17:58:52.071830+0000","last_peered":"2024-03-16T17:58:52.071830+0000","last_clean":"2024-03-16T17:58:52.071830+0000","last_became_active":"2024-03-16T17:58:50.458381+0000","last_became_peered":"2024-03-16T17:58:50.458381+0000","last_unstale":"2024-03-16T17:58:52.071830+0000","last_undegraded":"2024-03-16T17:58:52.071830+0000","last_fullsized":"2024-03-16T17:58:52.071830+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:18:17.056096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071866+0000","last_change":"2024-03-16T17:58:50.468540+0000","last_active":"2024-03-16T17:58:52.071866+0000","last_peered":"2024-03-16T17:58:52.071866+0000","last_clean":"2024-03-16T17:58:52.071866+0000","last_became_active":"2024-03-16T17:58:50.468304+0000","last_became_peered":"2024-03-16T17:58:50.468304+0000","last_unstale":"2024-03-16T17:58:52.071866+0000","last_undegraded":"2024-03-16T17:58:52.071866+0000","last_fullsized":"2024-03-16T17:58:52.071866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:23:47.737703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070551+0000","last_change":"2024-03-16T17:58:50.466785+0000","last_active":"2024-03-16T17:58:52.070551+0000","last_peered":"2024-03-16T17:58:52.070551+0000","last_clean":"2024-03-16T17:58:52.070551+0000","last_became_active":"2024-03-16T17:58:50.466472+0000","last_became_peered":"2024-03-16T17:58:50.466472+0000","last_unstale":"2024-03-16T17:58:52.070551+0000","last_undegraded":"2024-03-16T17:58:52.070551+0000","last_fullsized":"2024-03-16T17:58:52.070551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:34:41.770525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454652+0000","last_change":"2024-03-16T17:58:50.457083+0000","last_active":"2024-03-16T17:58:51.454652+0000","last_peered":"2024-03-16T17:58:51.454652+0000","last_clean":"2024-03-16T17:58:51.454652+0000","last_became_active":"2024-03-16T17:58:50.456802+0000","last_became_peered":"2024-03-16T17:58:50.456802+0000","last_unstale":"2024-03-16T17:58:51.454652+0000","last_undegraded":"2024-03-16T17:58:51.454652+0000","last_fullsized":"2024-03-16T17:58:51.454652+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:35:50.299745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.072598+0000","last_change":"2024-03-16T17:58:49.445413+0000","last_active":"2024-03-16T17:58:52.072598+0000","last_peered":"2024-03-16T17:58:52.072598+0000","last_clean":"2024-03-16T17:58:52.072598+0000","last_became_active":"2024-03-16T17:58:49.444108+0000","last_became_peered":"2024-03-16T17:58:49.444108+0000","last_unstale":"2024-03-16T17:58:52.072598+0000","last_undegraded":"2024-03-16T17:58:52.072598+0000","last_fullsized":"2024-03-16T17:58:52.072598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:47:18.543862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.072485+0000","last_change":"2024-03-16T17:58:50.460640+0000","last_active":"2024-03-16T17:58:52.072485+0000","last_peered":"2024-03-16T17:58:52.072485+0000","last_clean":"2024-03-16T17:58:52.072485+0000","last_became_active":"2024-03-16T17:58:50.460353+0000","last_became_peered":"2024-03-16T17:58:50.460353+0000","last_unstale":"2024-03-16T17:58:52.072485+0000","last_undegraded":"2024-03-16T17:58:52.072485+0000","last_fullsized":"2024-03-16T17:58:52.072485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:28:15.828722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454809+0000","last_change":"2024-03-16T17:58:49.441656+0000","last_active":"2024-03-16T17:58:51.454809+0000","last_peered":"2024-03-16T17:58:51.454809+0000","last_clean":"2024-03-16T17:58:51.454809+0000","last_became_active":"2024-03-16T17:58:49.441381+0000","last_became_peered":"2024-03-16T17:58:49.441381+0000","last_unstale":"2024-03-16T17:58:51.454809+0000","last_undegraded":"2024-03-16T17:58:51.454809+0000","last_fullsized":"2024-03-16T17:58:51.454809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:43:55.619208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456759+0000","last_change":"2024-03-16T17:58:50.464683+0000","last_active":"2024-03-16T17:58:51.456759+0000","last_peered":"2024-03-16T17:58:51.456759+0000","last_clean":"2024-03-16T17:58:51.456759+0000","last_became_active":"2024-03-16T17:58:50.464420+0000","last_became_peered":"2024-03-16T17:58:50.464420+0000","last_unstale":"2024-03-16T17:58:51.456759+0000","last_undegraded":"2024-03-16T17:58:51.456759+0000","last_fullsized":"2024-03-16T17:58:51.456759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:45:02.201098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566413+0000","last_change":"2024-03-16T17:58:49.442968+0000","last_active":"2024-03-16T17:58:51.566413+0000","last_peered":"2024-03-16T17:58:51.566413+0000","last_clean":"2024-03-16T17:58:51.566413+0000","last_became_active":"2024-03-16T17:58:49.442640+0000","last_became_peered":"2024-03-16T17:58:49.442640+0000","last_unstale":"2024-03-16T17:58:51.566413+0000","last_undegraded":"2024-03-16T17:58:51.566413+0000","last_fullsized":"2024-03-16T17:58:51.566413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:55:48.317393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070822+0000","last_change":"2024-03-16T17:58:50.465385+0000","last_active":"2024-03-16T17:58:52.070822+0000","last_peered":"2024-03-16T17:58:52.070822+0000","last_clean":"2024-03-16T17:58:52.070822+0000","last_became_active":"2024-03-16T17:58:50.464970+0000","last_became_peered":"2024-03-16T17:58:50.464970+0000","last_unstale":"2024-03-16T17:58:52.070822+0000","last_undegraded":"2024-03-16T17:58:52.070822+0000","last_fullsized":"2024-03-16T17:58:52.070822+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:24:43.582545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455719+0000","last_change":"2024-03-16T17:58:49.443719+0000","last_active":"2024-03-16T17:58:51.455719+0000","last_peered":"2024-03-16T17:58:51.455719+0000","last_clean":"2024-03-16T17:58:51.455719+0000","last_became_active":"2024-03-16T17:58:49.441863+0000","last_became_peered":"2024-03-16T17:58:49.441863+0000","last_unstale":"2024-03-16T17:58:51.455719+0000","last_undegraded":"2024-03-16T17:58:51.455719+0000","last_fullsized":"2024-03-16T17:58:51.455719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:05:54.509792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.074020+0000","last_change":"2024-03-16T17:58:50.467649+0000","last_active":"2024-03-16T17:58:52.074020+0000","last_peered":"2024-03-16T17:58:52.074020+0000","last_clean":"2024-03-16T17:58:52.074020+0000","last_became_active":"2024-03-16T17:58:50.467255+0000","last_became_peered":"2024-03-16T17:58:50.467255+0000","last_unstale":"2024-03-16T17:58:52.074020+0000","last_undegraded":"2024-03-16T17:58:52.074020+0000","last_fullsized":"2024-03-16T17:58:52.074020+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:19:18.993584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456705+0000","last_change":"2024-03-16T17:58:49.440885+0000","last_active":"2024-03-16T17:58:51.456705+0000","last_peered":"2024-03-16T17:58:51.456705+0000","last_clean":"2024-03-16T17:58:51.456705+0000","last_became_active":"2024-03-16T17:58:49.439408+0000","last_became_peered":"2024-03-16T17:58:49.439408+0000","last_unstale":"2024-03-16T17:58:51.456705+0000","last_undegraded":"2024-03-16T17:58:51.456705+0000","last_fullsized":"2024-03-16T17:58:51.456705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:03:35.750102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069271+0000","last_change":"2024-03-16T17:58:50.447195+0000","last_active":"2024-03-16T17:58:52.069271+0000","last_peered":"2024-03-16T17:58:52.069271+0000","last_clean":"2024-03-16T17:58:52.069271+0000","last_became_active":"2024-03-16T17:58:50.446866+0000","last_became_peered":"2024-03-16T17:58:50.446866+0000","last_unstale":"2024-03-16T17:58:52.069271+0000","last_undegraded":"2024-03-16T17:58:52.069271+0000","last_fullsized":"2024-03-16T17:58:52.069271+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:33:16.046811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456437+0000","last_change":"2024-03-16T17:58:49.434018+0000","last_active":"2024-03-16T17:58:51.456437+0000","last_peered":"2024-03-16T17:58:51.456437+0000","last_clean":"2024-03-16T17:58:51.456437+0000","last_became_active":"2024-03-16T17:58:49.433686+0000","last_became_peered":"2024-03-16T17:58:49.433686+0000","last_unstale":"2024-03-16T17:58:51.456437+0000","last_undegraded":"2024-03-16T17:58:51.456437+0000","last_fullsized":"2024-03-16T17:58:51.456437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:43:48.630048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455743+0000","last_change":"2024-03-16T17:58:50.445016+0000","last_active":"2024-03-16T17:58:51.455743+0000","last_peered":"2024-03-16T17:58:51.455743+0000","last_clean":"2024-03-16T17:58:51.455743+0000","last_became_active":"2024-03-16T17:58:50.444666+0000","last_became_peered":"2024-03-16T17:58:50.444666+0000","last_unstale":"2024-03-16T17:58:51.455743+0000","last_undegraded":"2024-03-16T17:58:51.455743+0000","last_fullsized":"2024-03-16T17:58:51.455743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:57:32.216661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.565875+0000","last_change":"2024-03-16T17:58:49.433715+0000","last_active":"2024-03-16T17:58:51.565875+0000","last_peered":"2024-03-16T17:58:51.565875+0000","last_clean":"2024-03-16T17:58:51.565875+0000","last_became_active":"2024-03-16T17:58:49.432360+0000","last_became_peered":"2024-03-16T17:58:49.432360+0000","last_unstale":"2024-03-16T17:58:51.565875+0000","last_undegraded":"2024-03-16T17:58:51.565875+0000","last_fullsized":"2024-03-16T17:58:51.565875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:28:14.815298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069676+0000","last_change":"2024-03-16T17:58:50.461819+0000","last_active":"2024-03-16T17:58:52.069676+0000","last_peered":"2024-03-16T17:58:52.069676+0000","last_clean":"2024-03-16T17:58:52.069676+0000","last_became_active":"2024-03-16T17:58:50.461510+0000","last_became_peered":"2024-03-16T17:58:50.461510+0000","last_unstale":"2024-03-16T17:58:52.069676+0000","last_undegraded":"2024-03-16T17:58:52.069676+0000","last_fullsized":"2024-03-16T17:58:52.069676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:57:21.130711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455490+0000","last_change":"2024-03-16T17:58:49.434253+0000","last_active":"2024-03-16T17:58:51.455490+0000","last_peered":"2024-03-16T17:58:51.455490+0000","last_clean":"2024-03-16T17:58:51.455490+0000","last_became_active":"2024-03-16T17:58:49.432867+0000","last_became_peered":"2024-03-16T17:58:49.432867+0000","last_unstale":"2024-03-16T17:58:51.455490+0000","last_undegraded":"2024-03-16T17:58:51.455490+0000","last_fullsized":"2024-03-16T17:58:51.455490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:55:34.304894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.073017+0000","last_change":"2024-03-16T17:58:50.442209+0000","last_active":"2024-03-16T17:58:52.073017+0000","last_peered":"2024-03-16T17:58:52.073017+0000","last_clean":"2024-03-16T17:58:52.073017+0000","last_became_active":"2024-03-16T17:58:50.441888+0000","last_became_peered":"2024-03-16T17:58:50.441888+0000","last_unstale":"2024-03-16T17:58:52.073017+0000","last_undegraded":"2024-03-16T17:58:52.073017+0000","last_fullsized":"2024-03-16T17:58:52.073017+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:12:09.875275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566661+0000","last_change":"2024-03-16T17:58:49.445682+0000","last_active":"2024-03-16T17:58:51.566661+0000","last_peered":"2024-03-16T17:58:51.566661+0000","last_clean":"2024-03-16T17:58:51.566661+0000","last_became_active":"2024-03-16T17:58:49.445395+0000","last_became_peered":"2024-03-16T17:58:49.445395+0000","last_unstale":"2024-03-16T17:58:51.566661+0000","last_undegraded":"2024-03-16T17:58:51.566661+0000","last_fullsized":"2024-03-16T17:58:51.566661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:44:20.723839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566575+0000","last_change":"2024-03-16T17:58:50.473505+0000","last_active":"2024-03-16T17:58:51.566575+0000","last_peered":"2024-03-16T17:58:51.566575+0000","last_clean":"2024-03-16T17:58:51.566575+0000","last_became_active":"2024-03-16T17:58:50.472581+0000","last_became_peered":"2024-03-16T17:58:50.472581+0000","last_unstale":"2024-03-16T17:58:51.566575+0000","last_undegraded":"2024-03-16T17:58:51.566575+0000","last_fullsized":"2024-03-16T17:58:51.566575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:03:27.488348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454916+0000","last_change":"2024-03-16T17:58:49.430438+0000","last_active":"2024-03-16T17:58:51.454916+0000","last_peered":"2024-03-16T17:58:51.454916+0000","last_clean":"2024-03-16T17:58:51.454916+0000","last_became_active":"2024-03-16T17:58:49.430168+0000","last_became_peered":"2024-03-16T17:58:49.430168+0000","last_unstale":"2024-03-16T17:58:51.454916+0000","last_undegraded":"2024-03-16T17:58:51.454916+0000","last_fullsized":"2024-03-16T17:58:51.454916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:47:18.543862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566513+0000","last_change":"2024-03-16T17:58:50.469214+0000","last_active":"2024-03-16T17:58:51.566513+0000","last_peered":"2024-03-16T17:58:51.566513+0000","last_clean":"2024-03-16T17:58:51.566513+0000","last_became_active":"2024-03-16T17:58:50.467172+0000","last_became_peered":"2024-03-16T17:58:50.467172+0000","last_unstale":"2024-03-16T17:58:51.566513+0000","last_undegraded":"2024-03-16T17:58:51.566513+0000","last_fullsized":"2024-03-16T17:58:51.566513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:03:24.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071393+0000","last_change":"2024-03-16T17:58:49.440525+0000","last_active":"2024-03-16T17:58:52.071393+0000","last_peered":"2024-03-16T17:58:52.071393+0000","last_clean":"2024-03-16T17:58:52.071393+0000","last_became_active":"2024-03-16T17:58:49.440023+0000","last_became_peered":"2024-03-16T17:58:49.440023+0000","last_unstale":"2024-03-16T17:58:52.071393+0000","last_undegraded":"2024-03-16T17:58:52.071393+0000","last_fullsized":"2024-03-16T17:58:52.071393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:32:25.953531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070983+0000","last_change":"2024-03-16T17:58:50.470586+0000","last_active":"2024-03-16T17:58:52.070983+0000","last_peered":"2024-03-16T17:58:52.070983+0000","last_clean":"2024-03-16T17:58:52.070983+0000","last_became_active":"2024-03-16T17:58:50.470246+0000","last_became_peered":"2024-03-16T17:58:50.470246+0000","last_unstale":"2024-03-16T17:58:52.070983+0000","last_undegraded":"2024-03-16T17:58:52.070983+0000","last_fullsized":"2024-03-16T17:58:52.070983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:03:24.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566595+0000","last_change":"2024-03-16T17:58:49.443184+0000","last_active":"2024-03-16T17:58:51.566595+0000","last_peered":"2024-03-16T17:58:51.566595+0000","last_clean":"2024-03-16T17:58:51.566595+0000","last_became_active":"2024-03-16T17:58:49.442622+0000","last_became_peered":"2024-03-16T17:58:49.442622+0000","last_unstale":"2024-03-16T17:58:51.566595+0000","last_undegraded":"2024-03-16T17:58:51.566595+0000","last_fullsized":"2024-03-16T17:58:51.566595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:35:49.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069578+0000","last_change":"2024-03-16T17:58:50.461790+0000","last_active":"2024-03-16T17:58:52.069578+0000","last_peered":"2024-03-16T17:58:52.069578+0000","last_clean":"2024-03-16T17:58:52.069578+0000","last_became_active":"2024-03-16T17:58:50.461409+0000","last_became_peered":"2024-03-16T17:58:50.461409+0000","last_unstale":"2024-03-16T17:58:52.069578+0000","last_undegraded":"2024-03-16T17:58:52.069578+0000","last_fullsized":"2024-03-16T17:58:52.069578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:32:26.966954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.457008+0000","last_change":"2024-03-16T17:58:49.439635+0000","last_active":"2024-03-16T17:58:51.457008+0000","last_peered":"2024-03-16T17:58:51.457008+0000","last_clean":"2024-03-16T17:58:51.457008+0000","last_became_active":"2024-03-16T17:58:49.439105+0000","last_became_peered":"2024-03-16T17:58:49.439105+0000","last_unstale":"2024-03-16T17:58:51.457008+0000","last_undegraded":"2024-03-16T17:58:51.457008+0000","last_fullsized":"2024-03-16T17:58:51.457008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:54:13.667841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566467+0000","last_change":"2024-03-16T17:58:50.451477+0000","last_active":"2024-03-16T17:58:51.566467+0000","last_peered":"2024-03-16T17:58:51.566467+0000","last_clean":"2024-03-16T17:58:51.566467+0000","last_became_active":"2024-03-16T17:58:50.450768+0000","last_became_peered":"2024-03-16T17:58:50.450768+0000","last_unstale":"2024-03-16T17:58:51.566467+0000","last_undegraded":"2024-03-16T17:58:51.566467+0000","last_fullsized":"2024-03-16T17:58:51.566467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:19:18.993584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069514+0000","last_change":"2024-03-16T17:58:49.446921+0000","last_active":"2024-03-16T17:58:52.069514+0000","last_peered":"2024-03-16T17:58:52.069514+0000","last_clean":"2024-03-16T17:58:52.069514+0000","last_became_active":"2024-03-16T17:58:49.445870+0000","last_became_peered":"2024-03-16T17:58:49.445870+0000","last_unstale":"2024-03-16T17:58:52.069514+0000","last_undegraded":"2024-03-16T17:58:52.069514+0000","last_fullsized":"2024-03-16T17:58:52.069514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:45:31.631750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070897+0000","last_change":"2024-03-16T17:58:50.473098+0000","last_active":"2024-03-16T17:58:52.070897+0000","last_peered":"2024-03-16T17:58:52.070897+0000","last_clean":"2024-03-16T17:58:52.070897+0000","last_became_active":"2024-03-16T17:58:50.472756+0000","last_became_peered":"2024-03-16T17:58:50.472756+0000","last_unstale":"2024-03-16T17:58:52.070897+0000","last_undegraded":"2024-03-16T17:58:52.070897+0000","last_fullsized":"2024-03-16T17:58:52.070897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:11:34.158762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.565759+0000","last_change":"2024-03-16T17:58:49.440799+0000","last_active":"2024-03-16T17:58:51.565759+0000","last_peered":"2024-03-16T17:58:51.565759+0000","last_clean":"2024-03-16T17:58:51.565759+0000","last_became_active":"2024-03-16T17:58:49.440432+0000","last_became_peered":"2024-03-16T17:58:49.440432+0000","last_unstale":"2024-03-16T17:58:51.565759+0000","last_undegraded":"2024-03-16T17:58:51.565759+0000","last_fullsized":"2024-03-16T17:58:51.565759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:53:50.195359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.072023+0000","last_change":"2024-03-16T17:58:50.459395+0000","last_active":"2024-03-16T17:58:52.072023+0000","last_peered":"2024-03-16T17:58:52.072023+0000","last_clean":"2024-03-16T17:58:52.072023+0000","last_became_active":"2024-03-16T17:58:50.459086+0000","last_became_peered":"2024-03-16T17:58:50.459086+0000","last_unstale":"2024-03-16T17:58:52.072023+0000","last_undegraded":"2024-03-16T17:58:52.072023+0000","last_fullsized":"2024-03-16T17:58:52.072023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:53:51.208782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566083+0000","last_change":"2024-03-16T17:58:49.443784+0000","last_active":"2024-03-16T17:58:51.566083+0000","last_peered":"2024-03-16T17:58:51.566083+0000","last_clean":"2024-03-16T17:58:51.566083+0000","last_became_active":"2024-03-16T17:58:49.442961+0000","last_became_peered":"2024-03-16T17:58:49.442961+0000","last_unstale":"2024-03-16T17:58:51.566083+0000","last_undegraded":"2024-03-16T17:58:51.566083+0000","last_fullsized":"2024-03-16T17:58:51.566083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:23:46.724279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.457396+0000","last_change":"2024-03-16T17:58:50.454631+0000","last_active":"2024-03-16T17:58:51.457396+0000","last_peered":"2024-03-16T17:58:51.457396+0000","last_clean":"2024-03-16T17:58:51.457396+0000","last_became_active":"2024-03-16T17:58:50.454233+0000","last_became_peered":"2024-03-16T17:58:50.454233+0000","last_unstale":"2024-03-16T17:58:51.457396+0000","last_undegraded":"2024-03-16T17:58:51.457396+0000","last_fullsized":"2024-03-16T17:58:51.457396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:50:54.165035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070698+0000","last_change":"2024-03-16T17:58:49.441265+0000","last_active":"2024-03-16T17:58:52.070698+0000","last_peered":"2024-03-16T17:58:52.070698+0000","last_clean":"2024-03-16T17:58:52.070698+0000","last_became_active":"2024-03-16T17:58:49.440840+0000","last_became_peered":"2024-03-16T17:58:49.440840+0000","last_unstale":"2024-03-16T17:58:52.070698+0000","last_undegraded":"2024-03-16T17:58:52.070698+0000","last_fullsized":"2024-03-16T17:58:52.070698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:53:50.195359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.072875+0000","last_change":"2024-03-16T17:58:50.454384+0000","last_active":"2024-03-16T17:58:52.072875+0000","last_peered":"2024-03-16T17:58:52.072875+0000","last_clean":"2024-03-16T17:58:52.072875+0000","last_became_active":"2024-03-16T17:58:50.454075+0000","last_became_peered":"2024-03-16T17:58:50.454075+0000","last_unstale":"2024-03-16T17:58:52.072875+0000","last_undegraded":"2024-03-16T17:58:52.072875+0000","last_fullsized":"2024-03-16T17:58:52.072875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:06:02.239701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.457615+0000","last_change":"2024-03-16T17:58:49.441789+0000","last_active":"2024-03-16T17:58:51.457615+0000","last_peered":"2024-03-16T17:58:51.457615+0000","last_clean":"2024-03-16T17:58:51.457615+0000","last_became_active":"2024-03-16T17:58:49.441384+0000","last_became_peered":"2024-03-16T17:58:49.441384+0000","last_unstale":"2024-03-16T17:58:51.457615+0000","last_undegraded":"2024-03-16T17:58:51.457615+0000","last_fullsized":"2024-03-16T17:58:51.457615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:41:19.921507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070654+0000","last_change":"2024-03-16T17:58:50.464530+0000","last_active":"2024-03-16T17:58:52.070654+0000","last_peered":"2024-03-16T17:58:52.070654+0000","last_clean":"2024-03-16T17:58:52.070654+0000","last_became_active":"2024-03-16T17:58:50.464194+0000","last_became_peered":"2024-03-16T17:58:50.464194+0000","last_unstale":"2024-03-16T17:58:52.070654+0000","last_undegraded":"2024-03-16T17:58:52.070654+0000","last_fullsized":"2024-03-16T17:58:52.070654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:00:53.339915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.457109+0000","last_change":"2024-03-16T17:58:49.442172+0000","last_active":"2024-03-16T17:58:51.457109+0000","last_peered":"2024-03-16T17:58:51.457109+0000","last_clean":"2024-03-16T17:58:51.457109+0000","last_became_active":"2024-03-16T17:58:49.441855+0000","last_became_peered":"2024-03-16T17:58:49.441855+0000","last_unstale":"2024-03-16T17:58:51.457109+0000","last_undegraded":"2024-03-16T17:58:51.457109+0000","last_fullsized":"2024-03-16T17:58:51.457109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:04:38.166526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455870+0000","last_change":"2024-03-16T17:58:50.473632+0000","last_active":"2024-03-16T17:58:51.455870+0000","last_peered":"2024-03-16T17:58:51.455870+0000","last_clean":"2024-03-16T17:58:51.455870+0000","last_became_active":"2024-03-16T17:58:50.473362+0000","last_became_peered":"2024-03-16T17:58:50.473362+0000","last_unstale":"2024-03-16T17:58:51.455870+0000","last_undegraded":"2024-03-16T17:58:51.455870+0000","last_fullsized":"2024-03-16T17:58:51.455870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:12:35.449102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456953+0000","last_change":"2024-03-16T17:58:49.439277+0000","last_active":"2024-03-16T17:58:51.456953+0000","last_peered":"2024-03-16T17:58:51.456953+0000","last_clean":"2024-03-16T17:58:51.456953+0000","last_became_active":"2024-03-16T17:58:49.438940+0000","last_became_peered":"2024-03-16T17:58:49.438940+0000","last_unstale":"2024-03-16T17:58:51.456953+0000","last_undegraded":"2024-03-16T17:58:51.456953+0000","last_fullsized":"2024-03-16T17:58:51.456953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:15:55.081410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455429+0000","last_change":"2024-03-16T17:58:50.469634+0000","last_active":"2024-03-16T17:58:51.455429+0000","last_peered":"2024-03-16T17:58:51.455429+0000","last_clean":"2024-03-16T17:58:51.455429+0000","last_became_active":"2024-03-16T17:58:50.469368+0000","last_became_peered":"2024-03-16T17:58:50.469368+0000","last_unstale":"2024-03-16T17:58:51.455429+0000","last_undegraded":"2024-03-16T17:58:51.455429+0000","last_fullsized":"2024-03-16T17:58:51.455429+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:41:20.934931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454434+0000","last_change":"2024-03-16T17:58:49.441933+0000","last_active":"2024-03-16T17:58:51.454434+0000","last_peered":"2024-03-16T17:58:51.454434+0000","last_clean":"2024-03-16T17:58:51.454434+0000","last_became_active":"2024-03-16T17:58:49.440983+0000","last_became_peered":"2024-03-16T17:58:49.440983+0000","last_unstale":"2024-03-16T17:58:51.454434+0000","last_undegraded":"2024-03-16T17:58:51.454434+0000","last_fullsized":"2024-03-16T17:58:51.454434+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:07:33.971257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455789+0000","last_change":"2024-03-16T17:58:50.464437+0000","last_active":"2024-03-16T17:58:51.455789+0000","last_peered":"2024-03-16T17:58:51.455789+0000","last_clean":"2024-03-16T17:58:51.455789+0000","last_became_active":"2024-03-16T17:58:50.464102+0000","last_became_peered":"2024-03-16T17:58:50.464102+0000","last_unstale":"2024-03-16T17:58:51.455789+0000","last_undegraded":"2024-03-16T17:58:51.455789+0000","last_fullsized":"2024-03-16T17:58:51.455789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:03:23.747692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071152+0000","last_change":"2024-03-16T17:58:49.445460+0000","last_active":"2024-03-16T17:58:52.071152+0000","last_peered":"2024-03-16T17:58:52.071152+0000","last_clean":"2024-03-16T17:58:52.071152+0000","last_became_active":"2024-03-16T17:58:49.445153+0000","last_became_peered":"2024-03-16T17:58:49.445153+0000","last_unstale":"2024-03-16T17:58:52.071152+0000","last_undegraded":"2024-03-16T17:58:52.071152+0000","last_fullsized":"2024-03-16T17:58:52.071152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:45:31.631750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454839+0000","last_change":"2024-03-16T17:58:50.464874+0000","last_active":"2024-03-16T17:58:51.454839+0000","last_peered":"2024-03-16T17:58:51.454839+0000","last_clean":"2024-03-16T17:58:51.454839+0000","last_became_active":"2024-03-16T17:58:50.463674+0000","last_became_peered":"2024-03-16T17:58:50.463674+0000","last_unstale":"2024-03-16T17:58:51.454839+0000","last_undegraded":"2024-03-16T17:58:51.454839+0000","last_fullsized":"2024-03-16T17:58:51.454839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:43:49.643472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455703+0000","last_change":"2024-03-16T17:58:49.442390+0000","last_active":"2024-03-16T17:58:51.455703+0000","last_peered":"2024-03-16T17:58:51.455703+0000","last_clean":"2024-03-16T17:58:51.455703+0000","last_became_active":"2024-03-16T17:58:49.442119+0000","last_became_peered":"2024-03-16T17:58:49.442119+0000","last_unstale":"2024-03-16T17:58:51.455703+0000","last_undegraded":"2024-03-16T17:58:51.455703+0000","last_fullsized":"2024-03-16T17:58:51.455703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:59:14.480003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.073504+0000","last_change":"2024-03-16T17:58:50.465615+0000","last_active":"2024-03-16T17:58:52.073504+0000","last_peered":"2024-03-16T17:58:52.073504+0000","last_clean":"2024-03-16T17:58:52.073504+0000","last_became_active":"2024-03-16T17:58:50.465291+0000","last_became_peered":"2024-03-16T17:58:50.465291+0000","last_unstale":"2024-03-16T17:58:52.073504+0000","last_undegraded":"2024-03-16T17:58:52.073504+0000","last_fullsized":"2024-03-16T17:58:52.073504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:14:57.330119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069398+0000","last_change":"2024-03-16T17:58:49.449370+0000","last_active":"2024-03-16T17:58:52.069398+0000","last_peered":"2024-03-16T17:58:52.069398+0000","last_clean":"2024-03-16T17:58:52.069398+0000","last_became_active":"2024-03-16T17:58:49.449033+0000","last_became_peered":"2024-03-16T17:58:49.449033+0000","last_unstale":"2024-03-16T17:58:52.069398+0000","last_undegraded":"2024-03-16T17:58:52.069398+0000","last_fullsized":"2024-03-16T17:58:52.069398+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:54:26.411849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456102+0000","last_change":"2024-03-16T17:58:50.466710+0000","last_active":"2024-03-16T17:58:51.456102+0000","last_peered":"2024-03-16T17:58:51.456102+0000","last_clean":"2024-03-16T17:58:51.456102+0000","last_became_active":"2024-03-16T17:58:50.466424+0000","last_became_peered":"2024-03-16T17:58:50.466424+0000","last_unstale":"2024-03-16T17:58:51.456102+0000","last_undegraded":"2024-03-16T17:58:51.456102+0000","last_fullsized":"2024-03-16T17:58:51.456102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:38:02.893836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070865+0000","last_change":"2024-03-16T17:58:49.441544+0000","last_active":"2024-03-16T17:58:52.070865+0000","last_peered":"2024-03-16T17:58:52.070865+0000","last_clean":"2024-03-16T17:58:52.070865+0000","last_became_active":"2024-03-16T17:58:49.440907+0000","last_became_peered":"2024-03-16T17:58:49.440907+0000","last_unstale":"2024-03-16T17:58:52.070865+0000","last_undegraded":"2024-03-16T17:58:52.070865+0000","last_fullsized":"2024-03-16T17:58:52.070865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:35:49.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.073296+0000","last_change":"2024-03-16T17:58:50.470549+0000","last_active":"2024-03-16T17:58:52.073296+0000","last_peered":"2024-03-16T17:58:52.073296+0000","last_clean":"2024-03-16T17:58:52.073296+0000","last_became_active":"2024-03-16T17:58:50.470269+0000","last_became_peered":"2024-03-16T17:58:50.470269+0000","last_unstale":"2024-03-16T17:58:52.073296+0000","last_undegraded":"2024-03-16T17:58:52.073296+0000","last_fullsized":"2024-03-16T17:58:52.073296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:18:17.534434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455038+0000","last_change":"2024-03-16T17:58:49.437446+0000","last_active":"2024-03-16T17:58:51.455038+0000","last_peered":"2024-03-16T17:58:51.455038+0000","last_clean":"2024-03-16T17:58:51.455038+0000","last_became_active":"2024-03-16T17:58:49.437170+0000","last_became_peered":"2024-03-16T17:58:49.437170+0000","last_unstale":"2024-03-16T17:58:51.455038+0000","last_undegraded":"2024-03-16T17:58:51.455038+0000","last_fullsized":"2024-03-16T17:58:51.455038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:45:31.631750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454765+0000","last_change":"2024-03-16T17:58:50.462471+0000","last_active":"2024-03-16T17:58:51.454765+0000","last_peered":"2024-03-16T17:58:51.454765+0000","last_clean":"2024-03-16T17:58:51.454765+0000","last_became_active":"2024-03-16T17:58:50.462123+0000","last_became_peered":"2024-03-16T17:58:50.462123+0000","last_unstale":"2024-03-16T17:58:51.454765+0000","last_undegraded":"2024-03-16T17:58:51.454765+0000","last_fullsized":"2024-03-16T17:58:51.454765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:28:15.828722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070535+0000","last_change":"2024-03-16T17:58:49.441844+0000","last_active":"2024-03-16T17:58:52.070535+0000","last_peered":"2024-03-16T17:58:52.070535+0000","last_clean":"2024-03-16T17:58:52.070535+0000","last_became_active":"2024-03-16T17:58:49.441548+0000","last_became_peered":"2024-03-16T17:58:49.441548+0000","last_unstale":"2024-03-16T17:58:52.070535+0000","last_undegraded":"2024-03-16T17:58:52.070535+0000","last_fullsized":"2024-03-16T17:58:52.070535+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:13:38.431719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071010+0000","last_change":"2024-03-16T17:58:50.451937+0000","last_active":"2024-03-16T17:58:52.071010+0000","last_peered":"2024-03-16T17:58:52.071010+0000","last_clean":"2024-03-16T17:58:52.071010+0000","last_became_active":"2024-03-16T17:58:50.451623+0000","last_became_peered":"2024-03-16T17:58:50.451623+0000","last_unstale":"2024-03-16T17:58:52.071010+0000","last_undegraded":"2024-03-16T17:58:52.071010+0000","last_fullsized":"2024-03-16T17:58:52.071010+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:21:37.094237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454745+0000","last_change":"2024-03-16T17:58:49.436289+0000","last_active":"2024-03-16T17:58:51.454745+0000","last_peered":"2024-03-16T17:58:51.454745+0000","last_clean":"2024-03-16T17:58:51.454745+0000","last_became_active":"2024-03-16T17:58:49.435849+0000","last_became_peered":"2024-03-16T17:58:49.435849+0000","last_unstale":"2024-03-16T17:58:51.454745+0000","last_undegraded":"2024-03-16T17:58:51.454745+0000","last_fullsized":"2024-03-16T17:58:51.454745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:13:38.431719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454613+0000","last_change":"2024-03-16T17:58:50.454567+0000","last_active":"2024-03-16T17:58:51.454613+0000","last_peered":"2024-03-16T17:58:51.454613+0000","last_clean":"2024-03-16T17:58:51.454613+0000","last_became_active":"2024-03-16T17:58:50.454220+0000","last_became_peered":"2024-03-16T17:58:50.454220+0000","last_unstale":"2024-03-16T17:58:51.454613+0000","last_undegraded":"2024-03-16T17:58:51.454613+0000","last_fullsized":"2024-03-16T17:58:51.454613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:53:51.208782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069816+0000","last_change":"2024-03-16T17:58:49.445120+0000","last_active":"2024-03-16T17:58:52.069816+0000","last_peered":"2024-03-16T17:58:52.069816+0000","last_clean":"2024-03-16T17:58:52.069816+0000","last_became_active":"2024-03-16T17:58:49.444850+0000","last_became_peered":"2024-03-16T17:58:49.444850+0000","last_unstale":"2024-03-16T17:58:52.069816+0000","last_undegraded":"2024-03-16T17:58:52.069816+0000","last_fullsized":"2024-03-16T17:58:52.069816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:07:33.971257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.565858+0000","last_change":"2024-03-16T17:58:50.448159+0000","last_active":"2024-03-16T17:58:51.565858+0000","last_peered":"2024-03-16T17:58:51.565858+0000","last_clean":"2024-03-16T17:58:51.565858+0000","last_became_active":"2024-03-16T17:58:50.447843+0000","last_became_peered":"2024-03-16T17:58:50.447843+0000","last_unstale":"2024-03-16T17:58:51.565858+0000","last_undegraded":"2024-03-16T17:58:51.565858+0000","last_fullsized":"2024-03-16T17:58:51.565858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:34:41.770525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456085+0000","last_change":"2024-03-16T17:58:49.441292+0000","last_active":"2024-03-16T17:58:51.456085+0000","last_peered":"2024-03-16T17:58:51.456085+0000","last_clean":"2024-03-16T17:58:51.456085+0000","last_became_active":"2024-03-16T17:58:49.440973+0000","last_became_peered":"2024-03-16T17:58:49.440973+0000","last_unstale":"2024-03-16T17:58:51.456085+0000","last_undegraded":"2024-03-16T17:58:51.456085+0000","last_fullsized":"2024-03-16T17:58:51.456085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:38:13.871717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455472+0000","last_change":"2024-03-16T17:58:50.448187+0000","last_active":"2024-03-16T17:58:51.455472+0000","last_peered":"2024-03-16T17:58:51.455472+0000","last_clean":"2024-03-16T17:58:51.455472+0000","last_became_active":"2024-03-16T17:58:50.447828+0000","last_became_peered":"2024-03-16T17:58:50.447828+0000","last_unstale":"2024-03-16T17:58:51.455472+0000","last_undegraded":"2024-03-16T17:58:51.455472+0000","last_fullsized":"2024-03-16T17:58:51.455472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:46:44.667447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.565906+0000","last_change":"2024-03-16T17:58:49.444926+0000","last_active":"2024-03-16T17:58:51.565906+0000","last_peered":"2024-03-16T17:58:51.565906+0000","last_clean":"2024-03-16T17:58:51.565906+0000","last_became_active":"2024-03-16T17:58:49.444648+0000","last_became_peered":"2024-03-16T17:58:49.444648+0000","last_unstale":"2024-03-16T17:58:51.565906+0000","last_undegraded":"2024-03-16T17:58:51.565906+0000","last_fullsized":"2024-03-16T17:58:51.565906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:21:36.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.074091+0000","last_change":"2024-03-16T17:58:50.468967+0000","last_active":"2024-03-16T17:58:52.074091+0000","last_peered":"2024-03-16T17:58:52.074091+0000","last_clean":"2024-03-16T17:58:52.074091+0000","last_became_active":"2024-03-16T17:58:50.468485+0000","last_became_peered":"2024-03-16T17:58:50.468485+0000","last_unstale":"2024-03-16T17:58:52.074091+0000","last_undegraded":"2024-03-16T17:58:52.074091+0000","last_fullsized":"2024-03-16T17:58:52.074091+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:03:24.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070457+0000","last_change":"2024-03-16T17:58:49.444380+0000","last_active":"2024-03-16T17:58:52.070457+0000","last_peered":"2024-03-16T17:58:52.070457+0000","last_clean":"2024-03-16T17:58:52.070457+0000","last_became_active":"2024-03-16T17:58:49.444051+0000","last_became_peered":"2024-03-16T17:58:49.444051+0000","last_unstale":"2024-03-16T17:58:52.070457+0000","last_undegraded":"2024-03-16T17:58:52.070457+0000","last_fullsized":"2024-03-16T17:58:52.070457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:43:55.619208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071241+0000","last_change":"2024-03-16T17:57:35.560172+0000","last_active":"2024-03-16T17:58:52.071241+0000","last_peered":"2024-03-16T17:58:52.071241+0000","last_clean":"2024-03-16T17:58:52.071241+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T17:58:52.071241+0000","last_undegraded":"2024-03-16T17:58:52.071241+0000","last_fullsized":"2024-03-16T17:58:52.071241+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_clean_scrub_stamp":"2024-03-16T17:57:34.174437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:20:03.039404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":740465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":151641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":150077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":151641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":740375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T17:58:54.896 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T17:58:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.141+0000 7fbb7a000640 1 Processor -- start 2024-03-16T17:58:55.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.142+0000 7fbb7a000640 1 -- start start 2024-03-16T17:58:55.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.143+0000 7fbb7a000640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fbb741595b0 0x7fbb740ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.143+0000 7fbb7a000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740b3370 con 0x7fbb74159990 2024-03-16T17:58:55.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.143+0000 7fbb7a000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7406dd20 con 0x7fbb7406d740 2024-03-16T17:58:55.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.143+0000 7fbb7a000640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7415f7a0 con 0x7fbb741595b0 2024-03-16T17:58:55.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.144+0000 7fbb6be00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbb74159990 0x7fbb7406d030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47778/0 (socket says 172.21.15.82:47778) 2024-03-16T17:58:55.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.144+0000 7fbb6be00640 1 -- 172.21.15.82:0/1017683890 learned_addr learned my addr 172.21.15.82:0/1017683890 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:55.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.144+0000 7fbb78c00640 1 --2- 172.21.15.82:0/1017683890 >> v2:172.21.15.90:3300/0 conn(0x7fbb741595b0 0x7fbb740ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:55.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.144+0000 7fbb73e00640 1 -- 172.21.15.82:0/1017683890 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455937228 0 0) 0x7fbb740b3370 con 0x7fbb74159990 2024-03-16T17:58:55.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.144+0000 7fbb73e00640 1 -- 172.21.15.82:0/1017683890 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb58003520 con 0x7fbb74159990 2024-03-16T17:58:55.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.145+0000 7fbb73e00640 1 -- 172.21.15.82:0/1017683890 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 288132000 0 0) 0x7fbb58003520 con 0x7fbb74159990 2024-03-16T17:58:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.145+0000 7fbb78c00640 1 -- 172.21.15.82:0/1017683890 >> v1:172.21.15.90:6789/0 conn(0x7fbb7406d740 legacy=0x7fbb7415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.145+0000 7fbb78c00640 1 -- 172.21.15.82:0/1017683890 >> v1:172.21.15.82:6789/0 conn(0x7fbb74159990 legacy=0x7fbb7406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.145+0000 7fbb78c00640 1 -- 172.21.15.82:0/1017683890 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7415fa80 con 0x7fbb741595b0 2024-03-16T17:58:55.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.146+0000 7fbb78c00640 1 --2- 172.21.15.82:0/1017683890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb741595b0 0x7fbb740ae730 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbb640097a0 tx=0x7fbb6402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8154af97b857987f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:55.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.146+0000 7fbb73e00640 1 -- 172.21.15.82:0/1017683890 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c001670 con 0x7fbb741595b0 2024-03-16T17:58:55.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.146+0000 7fbb73e00640 1 -- 172.21.15.82:0/1017683890 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6402e050 con 0x7fbb741595b0 2024-03-16T17:58:55.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.146+0000 7fbb73e00640 1 -- 172.21.15.82:0/1017683890 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64030710 con 0x7fbb741595b0 2024-03-16T17:58:55.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.147+0000 7fbb7a000640 1 -- 172.21.15.82:0/1017683890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb741595b0 msgr2=0x7fbb740ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.147+0000 7fbb7a000640 1 --2- 172.21.15.82:0/1017683890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb741595b0 0x7fbb740ae730 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbb640097a0 tx=0x7fbb6402b790 comp rx=0 tx=0).stop 2024-03-16T17:58:55.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.147+0000 7fbb7a000640 1 -- 172.21.15.82:0/1017683890 shutdown_connections 2024-03-16T17:58:55.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.147+0000 7fbb7a000640 1 --2- 172.21.15.82:0/1017683890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb741595b0 0x7fbb740ae730 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.147+0000 7fbb7a000640 1 -- 172.21.15.82:0/1017683890 >> 172.21.15.82:0/1017683890 conn(0x7fbb7406ae70 msgr2=0x7fbb7406b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:55.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.147+0000 7fbb7a000640 1 -- 172.21.15.82:0/1017683890 shutdown_connections 2024-03-16T17:58:55.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.147+0000 7fbb7a000640 1 -- 172.21.15.82:0/1017683890 wait complete. 2024-03-16T17:58:55.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.149+0000 7fbb7a000640 1 Processor -- start 2024-03-16T17:58:55.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.149+0000 7fbb7a000640 1 -- start start 2024-03-16T17:58:55.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.150+0000 7fbb7a000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7406d740 0x7fbb74149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.150+0000 7fbb7a000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb741595b0 0x7fbb740ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.150+0000 7fbb7a000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbb74159990 0x7fbb740add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.150+0000 7fbb7a000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb741f25f0 con 0x7fbb7406d740 2024-03-16T17:58:55.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.150+0000 7fbb7a000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74153760 con 0x7fbb741595b0 2024-03-16T17:58:55.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.150+0000 7fbb7a000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb741f1c90 con 0x7fbb74159990 2024-03-16T17:58:55.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.150+0000 7fbb78c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7406d740 0x7fbb74149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.150+0000 7fbb78c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7406d740 0x7fbb74149320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37938/0 (socket says 172.21.15.82:37938) 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.150+0000 7fbb78c00640 1 -- 172.21.15.82:0/2185430206 learned_addr learned my addr 172.21.15.82:0/2185430206 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb6be00640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb741595b0 0x7fbb740ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb78c00640 1 -- 172.21.15.82:0/2185430206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbb74159990 msgr2=0x7fbb740add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb79600640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbb74159990 0x7fbb740add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb78c00640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbb74159990 0x7fbb740add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb78c00640 1 -- 172.21.15.82:0/2185430206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb741595b0 msgr2=0x7fbb740ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb78c00640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb741595b0 0x7fbb740ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb78c00640 1 -- 172.21.15.82:0/2185430206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb64009400 con 0x7fbb7406d740 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb6be00640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb741595b0 0x7fbb740ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb78c00640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7406d740 0x7fbb74149320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbb6402b7d0 tx=0x7fbb640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb71600640 1 -- 172.21.15.82:0/2185430206 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64030710 con 0x7fbb7406d740 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb71600640 1 -- 172.21.15.82:0/2185430206 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb64030870 con 0x7fbb7406d740 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb7a000640 1 -- 172.21.15.82:0/2185430206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb741606e0 con 0x7fbb7406d740 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb71600640 1 -- 172.21.15.82:0/2185430206 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64030c20 con 0x7fbb7406d740 2024-03-16T17:58:55.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.151+0000 7fbb7a000640 1 -- 172.21.15.82:0/2185430206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb741f2210 con 0x7fbb7406d740 2024-03-16T17:58:55.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.152+0000 7fbb7a000640 1 -- 172.21.15.82:0/2185430206 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb741f2730 con 0x7fbb7406d740 2024-03-16T17:58:55.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.153+0000 7fbb71600640 1 -- 172.21.15.82:0/2185430206 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fbb64039050 con 0x7fbb7406d740 2024-03-16T17:58:55.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.154+0000 7fbb71600640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbb5006b3b0 0x7fbb5006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.154+0000 7fbb71600640 1 -- 172.21.15.82:0/2185430206 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb640ab690 con 0x7fbb7406d740 2024-03-16T17:58:55.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.154+0000 7fbb6be00640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbb5006b3b0 0x7fbb5006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:55.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.155+0000 7fbb6be00640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbb5006b3b0 0x7fbb5006d870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb74153bd0 tx=0x7fbb5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:55.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.159+0000 7fbb71600640 1 -- 172.21.15.82:0/2185430206 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbb64076380 con 0x7fbb7406d740 2024-03-16T17:58:55.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.351+0000 7fbb7a000640 1 -- 172.21.15.82:0/2185430206 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb74160040 con 0x7fbb5006b3b0 2024-03-16T17:58:55.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.370+0000 7fbb71600640 1 -- 172.21.15.82:0/2185430206 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251950 (secure 0 0 0) 0x7fbb5c0058d0 con 0x7fbb5006b3b0 2024-03-16T17:58:55.377 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:55.378 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T17:58:55.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 -- 172.21.15.82:0/2185430206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbb5006b3b0 msgr2=0x7fbb5006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbb5006b3b0 0x7fbb5006d870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb74153bd0 tx=0x7fbb5c009040 comp rx=0 tx=0).stop 2024-03-16T17:58:55.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 -- 172.21.15.82:0/2185430206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7406d740 msgr2=0x7fbb74149320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7406d740 0x7fbb74149320 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbb6402b7d0 tx=0x7fbb640046c0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 -- 172.21.15.82:0/2185430206 shutdown_connections 2024-03-16T17:58:55.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fbb5006b3b0 0x7fbb5006d870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fbb74159990 0x7fbb740add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb741595b0 0x7fbb740ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 --2- 172.21.15.82:0/2185430206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7406d740 0x7fbb74149320 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 -- 172.21.15.82:0/2185430206 >> 172.21.15.82:0/2185430206 conn(0x7fbb7406ae70 msgr2=0x7fbb740cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:55.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 -- 172.21.15.82:0/2185430206 shutdown_connections 2024-03-16T17:58:55.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.375+0000 7fbb7a000640 1 -- 172.21.15.82:0/2185430206 wait complete. 2024-03-16T17:58:55.398 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-03-16T17:58:54.033327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220396,"kb_used_data":4200,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754324,"statfs":{"total":773094113280,"available":772868427776,"internally_reserved":0,"allocated":4300800,"data_stored":2385612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.632590"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.072327+0000","last_change":"2024-03-16T17:58:50.463404+0000","last_active":"2024-03-16T17:58:52.072327+0000","last_peered":"2024-03-16T17:58:52.072327+0000","last_clean":"2024-03-16T17:58:52.072327+0000","last_became_active":"2024-03-16T17:58:50.463099+0000","last_became_peered":"2024-03-16T17:58:50.463099+0000","last_unstale":"2024-03-16T17:58:52.072327+0000","last_undegraded":"2024-03-16T17:58:52.072327+0000","last_fullsized":"2024-03-16T17:58:52.072327+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:35:50.299745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566333+0000","last_change":"2024-03-16T17:58:50.452964+0000","last_active":"2024-03-16T17:58:51.566333+0000","last_peered":"2024-03-16T17:58:51.566333+0000","last_clean":"2024-03-16T17:58:51.566333+0000","last_became_active":"2024-03-16T17:58:50.452700+0000","last_became_peered":"2024-03-16T17:58:50.452700+0000","last_unstale":"2024-03-16T17:58:51.566333+0000","last_undegraded":"2024-03-16T17:58:51.566333+0000","last_fullsized":"2024-03-16T17:58:51.566333+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:14:57.330119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071274+0000","last_change":"2024-03-16T17:58:50.461858+0000","last_active":"2024-03-16T17:58:52.071274+0000","last_peered":"2024-03-16T17:58:52.071274+0000","last_clean":"2024-03-16T17:58:52.071274+0000","last_became_active":"2024-03-16T17:58:50.461515+0000","last_became_peered":"2024-03-16T17:58:50.461515+0000","last_unstale":"2024-03-16T17:58:52.071274+0000","last_undegraded":"2024-03-16T17:58:52.071274+0000","last_fullsized":"2024-03-16T17:58:52.071274+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:03:27.488348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070236+0000","last_change":"2024-03-16T17:58:50.464429+0000","last_active":"2024-03-16T17:58:52.070236+0000","last_peered":"2024-03-16T17:58:52.070236+0000","last_clean":"2024-03-16T17:58:52.070236+0000","last_became_active":"2024-03-16T17:58:50.464155+0000","last_became_peered":"2024-03-16T17:58:50.464155+0000","last_unstale":"2024-03-16T17:58:52.070236+0000","last_undegraded":"2024-03-16T17:58:52.070236+0000","last_fullsized":"2024-03-16T17:58:52.070236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:46:45.157408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455241+0000","last_change":"2024-03-16T17:58:50.475001+0000","last_active":"2024-03-16T17:58:51.455241+0000","last_peered":"2024-03-16T17:58:51.455241+0000","last_clean":"2024-03-16T17:58:51.455241+0000","last_became_active":"2024-03-16T17:58:50.474725+0000","last_became_peered":"2024-03-16T17:58:50.474725+0000","last_unstale":"2024-03-16T17:58:51.455241+0000","last_undegraded":"2024-03-16T17:58:51.455241+0000","last_fullsized":"2024-03-16T17:58:51.455241+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:15:56.094834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.565988+0000","last_change":"2024-03-16T17:58:50.444215+0000","last_active":"2024-03-16T17:58:51.565988+0000","last_peered":"2024-03-16T17:58:51.565988+0000","last_clean":"2024-03-16T17:58:51.565988+0000","last_became_active":"2024-03-16T17:58:50.443676+0000","last_became_peered":"2024-03-16T17:58:50.443676+0000","last_unstale":"2024-03-16T17:58:51.565988+0000","last_undegraded":"2024-03-16T17:58:51.565988+0000","last_fullsized":"2024-03-16T17:58:51.565988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:11:34.158762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070019+0000","last_change":"2024-03-16T17:58:50.459374+0000","last_active":"2024-03-16T17:58:52.070019+0000","last_peered":"2024-03-16T17:58:52.070019+0000","last_clean":"2024-03-16T17:58:52.070019+0000","last_became_active":"2024-03-16T17:58:50.459010+0000","last_became_peered":"2024-03-16T17:58:50.459010+0000","last_unstale":"2024-03-16T17:58:52.070019+0000","last_undegraded":"2024-03-16T17:58:52.070019+0000","last_fullsized":"2024-03-16T17:58:52.070019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:53:55.960247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069948+0000","last_change":"2024-03-16T17:58:50.468977+0000","last_active":"2024-03-16T17:58:52.069948+0000","last_peered":"2024-03-16T17:58:52.069948+0000","last_clean":"2024-03-16T17:58:52.069948+0000","last_became_active":"2024-03-16T17:58:50.468622+0000","last_became_peered":"2024-03-16T17:58:50.468622+0000","last_unstale":"2024-03-16T17:58:52.069948+0000","last_undegraded":"2024-03-16T17:58:52.069948+0000","last_fullsized":"2024-03-16T17:58:52.069948+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:04:31.126786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.457060+0000","last_change":"2024-03-16T17:58:50.475809+0000","last_active":"2024-03-16T17:58:51.457060+0000","last_peered":"2024-03-16T17:58:51.457060+0000","last_clean":"2024-03-16T17:58:51.457060+0000","last_became_active":"2024-03-16T17:58:50.475419+0000","last_became_peered":"2024-03-16T17:58:50.475419+0000","last_unstale":"2024-03-16T17:58:51.457060+0000","last_undegraded":"2024-03-16T17:58:51.457060+0000","last_fullsized":"2024-03-16T17:58:51.457060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:28:50.473379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070335+0000","last_change":"2024-03-16T17:58:50.470761+0000","last_active":"2024-03-16T17:58:52.070335+0000","last_peered":"2024-03-16T17:58:52.070335+0000","last_clean":"2024-03-16T17:58:52.070335+0000","last_became_active":"2024-03-16T17:58:50.468543+0000","last_became_peered":"2024-03-16T17:58:50.468543+0000","last_unstale":"2024-03-16T17:58:52.070335+0000","last_undegraded":"2024-03-16T17:58:52.070335+0000","last_fullsized":"2024-03-16T17:58:52.070335+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:24:56.380097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.074163+0000","last_change":"2024-03-16T17:58:50.467169+0000","last_active":"2024-03-16T17:58:52.074163+0000","last_peered":"2024-03-16T17:58:52.074163+0000","last_clean":"2024-03-16T17:58:52.074163+0000","last_became_active":"2024-03-16T17:58:50.465308+0000","last_became_peered":"2024-03-16T17:58:50.465308+0000","last_unstale":"2024-03-16T17:58:52.074163+0000","last_undegraded":"2024-03-16T17:58:52.074163+0000","last_fullsized":"2024-03-16T17:58:52.074163+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:03:27.488348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069965+0000","last_change":"2024-03-16T17:58:50.442059+0000","last_active":"2024-03-16T17:58:52.069965+0000","last_peered":"2024-03-16T17:58:52.069965+0000","last_clean":"2024-03-16T17:58:52.069965+0000","last_became_active":"2024-03-16T17:58:50.439457+0000","last_became_peered":"2024-03-16T17:58:50.439457+0000","last_unstale":"2024-03-16T17:58:52.069965+0000","last_undegraded":"2024-03-16T17:58:52.069965+0000","last_fullsized":"2024-03-16T17:58:52.069965+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:14:57.330119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455470+0000","last_change":"2024-03-16T17:58:50.476284+0000","last_active":"2024-03-16T17:58:51.455470+0000","last_peered":"2024-03-16T17:58:51.455470+0000","last_clean":"2024-03-16T17:58:51.455470+0000","last_became_active":"2024-03-16T17:58:50.475925+0000","last_became_peered":"2024-03-16T17:58:50.475925+0000","last_unstale":"2024-03-16T17:58:51.455470+0000","last_undegraded":"2024-03-16T17:58:51.455470+0000","last_fullsized":"2024-03-16T17:58:51.455470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:15:14.236101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566080+0000","last_change":"2024-03-16T17:58:50.457152+0000","last_active":"2024-03-16T17:58:51.566080+0000","last_peered":"2024-03-16T17:58:51.566080+0000","last_clean":"2024-03-16T17:58:51.566080+0000","last_became_active":"2024-03-16T17:58:50.456805+0000","last_became_peered":"2024-03-16T17:58:50.456805+0000","last_unstale":"2024-03-16T17:58:51.566080+0000","last_undegraded":"2024-03-16T17:58:51.566080+0000","last_fullsized":"2024-03-16T17:58:51.566080+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:33:16.046811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069781+0000","last_change":"2024-03-16T17:58:50.446637+0000","last_active":"2024-03-16T17:58:52.069781+0000","last_peered":"2024-03-16T17:58:52.069781+0000","last_clean":"2024-03-16T17:58:52.069781+0000","last_became_active":"2024-03-16T17:58:50.446288+0000","last_became_peered":"2024-03-16T17:58:50.446288+0000","last_unstale":"2024-03-16T17:58:52.069781+0000","last_undegraded":"2024-03-16T17:58:52.069781+0000","last_fullsized":"2024-03-16T17:58:52.069781+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:18:17.056096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071357+0000","last_change":"2024-03-16T17:58:50.475384+0000","last_active":"2024-03-16T17:58:52.071357+0000","last_peered":"2024-03-16T17:58:52.071357+0000","last_clean":"2024-03-16T17:58:52.071357+0000","last_became_active":"2024-03-16T17:58:50.474977+0000","last_became_peered":"2024-03-16T17:58:50.474977+0000","last_unstale":"2024-03-16T17:58:52.071357+0000","last_undegraded":"2024-03-16T17:58:52.071357+0000","last_fullsized":"2024-03-16T17:58:52.071357+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:33:16.046811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.074586+0000","last_change":"2024-03-16T17:58:50.467327+0000","last_active":"2024-03-16T17:58:52.074586+0000","last_peered":"2024-03-16T17:58:52.074586+0000","last_clean":"2024-03-16T17:58:52.074586+0000","last_became_active":"2024-03-16T17:58:50.466879+0000","last_became_peered":"2024-03-16T17:58:50.466879+0000","last_unstale":"2024-03-16T17:58:52.074586+0000","last_undegraded":"2024-03-16T17:58:52.074586+0000","last_fullsized":"2024-03-16T17:58:52.074586+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:06:02.239701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566715+0000","last_change":"2024-03-16T17:58:50.462782+0000","last_active":"2024-03-16T17:58:51.566715+0000","last_peered":"2024-03-16T17:58:51.566715+0000","last_clean":"2024-03-16T17:58:51.566715+0000","last_became_active":"2024-03-16T17:58:50.462504+0000","last_became_peered":"2024-03-16T17:58:50.462504+0000","last_unstale":"2024-03-16T17:58:51.566715+0000","last_undegraded":"2024-03-16T17:58:51.566715+0000","last_fullsized":"2024-03-16T17:58:51.566715+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:06:02.239701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566298+0000","last_change":"2024-03-16T17:58:50.463054+0000","last_active":"2024-03-16T17:58:51.566298+0000","last_peered":"2024-03-16T17:58:51.566298+0000","last_clean":"2024-03-16T17:58:51.566298+0000","last_became_active":"2024-03-16T17:58:50.462563+0000","last_became_peered":"2024-03-16T17:58:50.462563+0000","last_unstale":"2024-03-16T17:58:51.566298+0000","last_undegraded":"2024-03-16T17:58:51.566298+0000","last_fullsized":"2024-03-16T17:58:51.566298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:18:17.056096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455736+0000","last_change":"2024-03-16T17:58:50.465327+0000","last_active":"2024-03-16T17:58:51.455736+0000","last_peered":"2024-03-16T17:58:51.455736+0000","last_clean":"2024-03-16T17:58:51.455736+0000","last_became_active":"2024-03-16T17:58:50.463433+0000","last_became_peered":"2024-03-16T17:58:50.463433+0000","last_unstale":"2024-03-16T17:58:51.455736+0000","last_undegraded":"2024-03-16T17:58:51.455736+0000","last_fullsized":"2024-03-16T17:58:51.455736+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:33.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456464+0000","last_change":"2024-03-16T17:58:50.448625+0000","last_active":"2024-03-16T17:58:51.456464+0000","last_peered":"2024-03-16T17:58:51.456464+0000","last_clean":"2024-03-16T17:58:51.456464+0000","last_became_active":"2024-03-16T17:58:50.448235+0000","last_became_peered":"2024-03-16T17:58:50.448235+0000","last_unstale":"2024-03-16T17:58:51.456464+0000","last_undegraded":"2024-03-16T17:58:51.456464+0000","last_fullsized":"2024-03-16T17:58:51.456464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:22:41.995233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456489+0000","last_change":"2024-03-16T17:58:50.446583+0000","last_active":"2024-03-16T17:58:51.456489+0000","last_peered":"2024-03-16T17:58:51.456489+0000","last_clean":"2024-03-16T17:58:51.456489+0000","last_became_active":"2024-03-16T17:58:50.446296+0000","last_became_peered":"2024-03-16T17:58:50.446296+0000","last_unstale":"2024-03-16T17:58:51.456489+0000","last_undegraded":"2024-03-16T17:58:51.456489+0000","last_fullsized":"2024-03-16T17:58:51.456489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:21:28.840567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.453952+0000","last_change":"2024-03-16T17:58:50.456867+0000","last_active":"2024-03-16T17:58:51.453952+0000","last_peered":"2024-03-16T17:58:51.453952+0000","last_clean":"2024-03-16T17:58:51.453952+0000","last_became_active":"2024-03-16T17:58:50.456597+0000","last_became_peered":"2024-03-16T17:58:50.456597+0000","last_unstale":"2024-03-16T17:58:51.453952+0000","last_undegraded":"2024-03-16T17:58:51.453952+0000","last_fullsized":"2024-03-16T17:58:51.453952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:23:47.737703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069116+0000","last_change":"2024-03-16T17:58:50.472775+0000","last_active":"2024-03-16T17:58:52.069116+0000","last_peered":"2024-03-16T17:58:52.069116+0000","last_clean":"2024-03-16T17:58:52.069116+0000","last_became_active":"2024-03-16T17:58:50.472197+0000","last_became_peered":"2024-03-16T17:58:50.472197+0000","last_unstale":"2024-03-16T17:58:52.069116+0000","last_undegraded":"2024-03-16T17:58:52.069116+0000","last_fullsized":"2024-03-16T17:58:52.069116+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:11:34.158762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070506+0000","last_change":"2024-03-16T17:58:50.473039+0000","last_active":"2024-03-16T17:58:52.070506+0000","last_peered":"2024-03-16T17:58:52.070506+0000","last_clean":"2024-03-16T17:58:52.070506+0000","last_became_active":"2024-03-16T17:58:50.472760+0000","last_became_peered":"2024-03-16T17:58:50.472760+0000","last_unstale":"2024-03-16T17:58:52.070506+0000","last_undegraded":"2024-03-16T17:58:52.070506+0000","last_fullsized":"2024-03-16T17:58:52.070506+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:36:18.198333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566803+0000","last_change":"2024-03-16T17:58:50.467268+0000","last_active":"2024-03-16T17:58:51.566803+0000","last_peered":"2024-03-16T17:58:51.566803+0000","last_clean":"2024-03-16T17:58:51.566803+0000","last_became_active":"2024-03-16T17:58:50.466889+0000","last_became_peered":"2024-03-16T17:58:50.466889+0000","last_unstale":"2024-03-16T17:58:51.566803+0000","last_undegraded":"2024-03-16T17:58:51.566803+0000","last_fullsized":"2024-03-16T17:58:51.566803+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:12:09.875275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070686+0000","last_change":"2024-03-16T17:58:50.475482+0000","last_active":"2024-03-16T17:58:52.070686+0000","last_peered":"2024-03-16T17:58:52.070686+0000","last_clean":"2024-03-16T17:58:52.070686+0000","last_became_active":"2024-03-16T17:58:50.475185+0000","last_became_peered":"2024-03-16T17:58:50.475185+0000","last_unstale":"2024-03-16T17:58:52.070686+0000","last_undegraded":"2024-03-16T17:58:52.070686+0000","last_fullsized":"2024-03-16T17:58:52.070686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:48:12.585703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070224+0000","last_change":"2024-03-16T17:58:50.447944+0000","last_active":"2024-03-16T17:58:52.070224+0000","last_peered":"2024-03-16T17:58:52.070224+0000","last_clean":"2024-03-16T17:58:52.070224+0000","last_became_active":"2024-03-16T17:58:50.447395+0000","last_became_peered":"2024-03-16T17:58:50.447395+0000","last_unstale":"2024-03-16T17:58:52.070224+0000","last_undegraded":"2024-03-16T17:58:52.070224+0000","last_fullsized":"2024-03-16T17:58:52.070224+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:19:18.993584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456683+0000","last_change":"2024-03-16T17:58:50.455905+0000","last_active":"2024-03-16T17:58:51.456683+0000","last_peered":"2024-03-16T17:58:51.456683+0000","last_clean":"2024-03-16T17:58:51.456683+0000","last_became_active":"2024-03-16T17:58:50.454469+0000","last_became_peered":"2024-03-16T17:58:50.454469+0000","last_unstale":"2024-03-16T17:58:51.456683+0000","last_undegraded":"2024-03-16T17:58:51.456683+0000","last_fullsized":"2024-03-16T17:58:51.456683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:36:50.817760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071830+0000","last_change":"2024-03-16T17:58:50.458671+0000","last_active":"2024-03-16T17:58:52.071830+0000","last_peered":"2024-03-16T17:58:52.071830+0000","last_clean":"2024-03-16T17:58:52.071830+0000","last_became_active":"2024-03-16T17:58:50.458381+0000","last_became_peered":"2024-03-16T17:58:50.458381+0000","last_unstale":"2024-03-16T17:58:52.071830+0000","last_undegraded":"2024-03-16T17:58:52.071830+0000","last_fullsized":"2024-03-16T17:58:52.071830+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:18:17.056096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071866+0000","last_change":"2024-03-16T17:58:50.468540+0000","last_active":"2024-03-16T17:58:52.071866+0000","last_peered":"2024-03-16T17:58:52.071866+0000","last_clean":"2024-03-16T17:58:52.071866+0000","last_became_active":"2024-03-16T17:58:50.468304+0000","last_became_peered":"2024-03-16T17:58:50.468304+0000","last_unstale":"2024-03-16T17:58:52.071866+0000","last_undegraded":"2024-03-16T17:58:52.071866+0000","last_fullsized":"2024-03-16T17:58:52.071866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:23:47.737703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070551+0000","last_change":"2024-03-16T17:58:50.466785+0000","last_active":"2024-03-16T17:58:52.070551+0000","last_peered":"2024-03-16T17:58:52.070551+0000","last_clean":"2024-03-16T17:58:52.070551+0000","last_became_active":"2024-03-16T17:58:50.466472+0000","last_became_peered":"2024-03-16T17:58:50.466472+0000","last_unstale":"2024-03-16T17:58:52.070551+0000","last_undegraded":"2024-03-16T17:58:52.070551+0000","last_fullsized":"2024-03-16T17:58:52.070551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:34:41.770525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454652+0000","last_change":"2024-03-16T17:58:50.457083+0000","last_active":"2024-03-16T17:58:51.454652+0000","last_peered":"2024-03-16T17:58:51.454652+0000","last_clean":"2024-03-16T17:58:51.454652+0000","last_became_active":"2024-03-16T17:58:50.456802+0000","last_became_peered":"2024-03-16T17:58:50.456802+0000","last_unstale":"2024-03-16T17:58:51.454652+0000","last_undegraded":"2024-03-16T17:58:51.454652+0000","last_fullsized":"2024-03-16T17:58:51.454652+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:35:50.299745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.072598+0000","last_change":"2024-03-16T17:58:49.445413+0000","last_active":"2024-03-16T17:58:52.072598+0000","last_peered":"2024-03-16T17:58:52.072598+0000","last_clean":"2024-03-16T17:58:52.072598+0000","last_became_active":"2024-03-16T17:58:49.444108+0000","last_became_peered":"2024-03-16T17:58:49.444108+0000","last_unstale":"2024-03-16T17:58:52.072598+0000","last_undegraded":"2024-03-16T17:58:52.072598+0000","last_fullsized":"2024-03-16T17:58:52.072598+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:47:18.543862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.072485+0000","last_change":"2024-03-16T17:58:50.460640+0000","last_active":"2024-03-16T17:58:52.072485+0000","last_peered":"2024-03-16T17:58:52.072485+0000","last_clean":"2024-03-16T17:58:52.072485+0000","last_became_active":"2024-03-16T17:58:50.460353+0000","last_became_peered":"2024-03-16T17:58:50.460353+0000","last_unstale":"2024-03-16T17:58:52.072485+0000","last_undegraded":"2024-03-16T17:58:52.072485+0000","last_fullsized":"2024-03-16T17:58:52.072485+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:28:15.828722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454809+0000","last_change":"2024-03-16T17:58:49.441656+0000","last_active":"2024-03-16T17:58:51.454809+0000","last_peered":"2024-03-16T17:58:51.454809+0000","last_clean":"2024-03-16T17:58:51.454809+0000","last_became_active":"2024-03-16T17:58:49.441381+0000","last_became_peered":"2024-03-16T17:58:49.441381+0000","last_unstale":"2024-03-16T17:58:51.454809+0000","last_undegraded":"2024-03-16T17:58:51.454809+0000","last_fullsized":"2024-03-16T17:58:51.454809+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:43:55.619208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456759+0000","last_change":"2024-03-16T17:58:50.464683+0000","last_active":"2024-03-16T17:58:51.456759+0000","last_peered":"2024-03-16T17:58:51.456759+0000","last_clean":"2024-03-16T17:58:51.456759+0000","last_became_active":"2024-03-16T17:58:50.464420+0000","last_became_peered":"2024-03-16T17:58:50.464420+0000","last_unstale":"2024-03-16T17:58:51.456759+0000","last_undegraded":"2024-03-16T17:58:51.456759+0000","last_fullsized":"2024-03-16T17:58:51.456759+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:45:02.201098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566413+0000","last_change":"2024-03-16T17:58:49.442968+0000","last_active":"2024-03-16T17:58:51.566413+0000","last_peered":"2024-03-16T17:58:51.566413+0000","last_clean":"2024-03-16T17:58:51.566413+0000","last_became_active":"2024-03-16T17:58:49.442640+0000","last_became_peered":"2024-03-16T17:58:49.442640+0000","last_unstale":"2024-03-16T17:58:51.566413+0000","last_undegraded":"2024-03-16T17:58:51.566413+0000","last_fullsized":"2024-03-16T17:58:51.566413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:55:48.317393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070822+0000","last_change":"2024-03-16T17:58:50.465385+0000","last_active":"2024-03-16T17:58:52.070822+0000","last_peered":"2024-03-16T17:58:52.070822+0000","last_clean":"2024-03-16T17:58:52.070822+0000","last_became_active":"2024-03-16T17:58:50.464970+0000","last_became_peered":"2024-03-16T17:58:50.464970+0000","last_unstale":"2024-03-16T17:58:52.070822+0000","last_undegraded":"2024-03-16T17:58:52.070822+0000","last_fullsized":"2024-03-16T17:58:52.070822+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:24:43.582545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455719+0000","last_change":"2024-03-16T17:58:49.443719+0000","last_active":"2024-03-16T17:58:51.455719+0000","last_peered":"2024-03-16T17:58:51.455719+0000","last_clean":"2024-03-16T17:58:51.455719+0000","last_became_active":"2024-03-16T17:58:49.441863+0000","last_became_peered":"2024-03-16T17:58:49.441863+0000","last_unstale":"2024-03-16T17:58:51.455719+0000","last_undegraded":"2024-03-16T17:58:51.455719+0000","last_fullsized":"2024-03-16T17:58:51.455719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:05:54.509792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.074020+0000","last_change":"2024-03-16T17:58:50.467649+0000","last_active":"2024-03-16T17:58:52.074020+0000","last_peered":"2024-03-16T17:58:52.074020+0000","last_clean":"2024-03-16T17:58:52.074020+0000","last_became_active":"2024-03-16T17:58:50.467255+0000","last_became_peered":"2024-03-16T17:58:50.467255+0000","last_unstale":"2024-03-16T17:58:52.074020+0000","last_undegraded":"2024-03-16T17:58:52.074020+0000","last_fullsized":"2024-03-16T17:58:52.074020+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:19:18.993584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456705+0000","last_change":"2024-03-16T17:58:49.440885+0000","last_active":"2024-03-16T17:58:51.456705+0000","last_peered":"2024-03-16T17:58:51.456705+0000","last_clean":"2024-03-16T17:58:51.456705+0000","last_became_active":"2024-03-16T17:58:49.439408+0000","last_became_peered":"2024-03-16T17:58:49.439408+0000","last_unstale":"2024-03-16T17:58:51.456705+0000","last_undegraded":"2024-03-16T17:58:51.456705+0000","last_fullsized":"2024-03-16T17:58:51.456705+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:03:35.750102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069271+0000","last_change":"2024-03-16T17:58:50.447195+0000","last_active":"2024-03-16T17:58:52.069271+0000","last_peered":"2024-03-16T17:58:52.069271+0000","last_clean":"2024-03-16T17:58:52.069271+0000","last_became_active":"2024-03-16T17:58:50.446866+0000","last_became_peered":"2024-03-16T17:58:50.446866+0000","last_unstale":"2024-03-16T17:58:52.069271+0000","last_undegraded":"2024-03-16T17:58:52.069271+0000","last_fullsized":"2024-03-16T17:58:52.069271+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:33:16.046811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456437+0000","last_change":"2024-03-16T17:58:49.434018+0000","last_active":"2024-03-16T17:58:51.456437+0000","last_peered":"2024-03-16T17:58:51.456437+0000","last_clean":"2024-03-16T17:58:51.456437+0000","last_became_active":"2024-03-16T17:58:49.433686+0000","last_became_peered":"2024-03-16T17:58:49.433686+0000","last_unstale":"2024-03-16T17:58:51.456437+0000","last_undegraded":"2024-03-16T17:58:51.456437+0000","last_fullsized":"2024-03-16T17:58:51.456437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:43:48.630048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455743+0000","last_change":"2024-03-16T17:58:50.445016+0000","last_active":"2024-03-16T17:58:51.455743+0000","last_peered":"2024-03-16T17:58:51.455743+0000","last_clean":"2024-03-16T17:58:51.455743+0000","last_became_active":"2024-03-16T17:58:50.444666+0000","last_became_peered":"2024-03-16T17:58:50.444666+0000","last_unstale":"2024-03-16T17:58:51.455743+0000","last_undegraded":"2024-03-16T17:58:51.455743+0000","last_fullsized":"2024-03-16T17:58:51.455743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:57:32.216661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.565875+0000","last_change":"2024-03-16T17:58:49.433715+0000","last_active":"2024-03-16T17:58:51.565875+0000","last_peered":"2024-03-16T17:58:51.565875+0000","last_clean":"2024-03-16T17:58:51.565875+0000","last_became_active":"2024-03-16T17:58:49.432360+0000","last_became_peered":"2024-03-16T17:58:49.432360+0000","last_unstale":"2024-03-16T17:58:51.565875+0000","last_undegraded":"2024-03-16T17:58:51.565875+0000","last_fullsized":"2024-03-16T17:58:51.565875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:28:14.815298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069676+0000","last_change":"2024-03-16T17:58:50.461819+0000","last_active":"2024-03-16T17:58:52.069676+0000","last_peered":"2024-03-16T17:58:52.069676+0000","last_clean":"2024-03-16T17:58:52.069676+0000","last_became_active":"2024-03-16T17:58:50.461510+0000","last_became_peered":"2024-03-16T17:58:50.461510+0000","last_unstale":"2024-03-16T17:58:52.069676+0000","last_undegraded":"2024-03-16T17:58:52.069676+0000","last_fullsized":"2024-03-16T17:58:52.069676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:57:21.130711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455490+0000","last_change":"2024-03-16T17:58:49.434253+0000","last_active":"2024-03-16T17:58:51.455490+0000","last_peered":"2024-03-16T17:58:51.455490+0000","last_clean":"2024-03-16T17:58:51.455490+0000","last_became_active":"2024-03-16T17:58:49.432867+0000","last_became_peered":"2024-03-16T17:58:49.432867+0000","last_unstale":"2024-03-16T17:58:51.455490+0000","last_undegraded":"2024-03-16T17:58:51.455490+0000","last_fullsized":"2024-03-16T17:58:51.455490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:55:34.304894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.073017+0000","last_change":"2024-03-16T17:58:50.442209+0000","last_active":"2024-03-16T17:58:52.073017+0000","last_peered":"2024-03-16T17:58:52.073017+0000","last_clean":"2024-03-16T17:58:52.073017+0000","last_became_active":"2024-03-16T17:58:50.441888+0000","last_became_peered":"2024-03-16T17:58:50.441888+0000","last_unstale":"2024-03-16T17:58:52.073017+0000","last_undegraded":"2024-03-16T17:58:52.073017+0000","last_fullsized":"2024-03-16T17:58:52.073017+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:12:09.875275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566661+0000","last_change":"2024-03-16T17:58:49.445682+0000","last_active":"2024-03-16T17:58:51.566661+0000","last_peered":"2024-03-16T17:58:51.566661+0000","last_clean":"2024-03-16T17:58:51.566661+0000","last_became_active":"2024-03-16T17:58:49.445395+0000","last_became_peered":"2024-03-16T17:58:49.445395+0000","last_unstale":"2024-03-16T17:58:51.566661+0000","last_undegraded":"2024-03-16T17:58:51.566661+0000","last_fullsized":"2024-03-16T17:58:51.566661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:44:20.723839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566575+0000","last_change":"2024-03-16T17:58:50.473505+0000","last_active":"2024-03-16T17:58:51.566575+0000","last_peered":"2024-03-16T17:58:51.566575+0000","last_clean":"2024-03-16T17:58:51.566575+0000","last_became_active":"2024-03-16T17:58:50.472581+0000","last_became_peered":"2024-03-16T17:58:50.472581+0000","last_unstale":"2024-03-16T17:58:51.566575+0000","last_undegraded":"2024-03-16T17:58:51.566575+0000","last_fullsized":"2024-03-16T17:58:51.566575+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:03:27.488348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454916+0000","last_change":"2024-03-16T17:58:49.430438+0000","last_active":"2024-03-16T17:58:51.454916+0000","last_peered":"2024-03-16T17:58:51.454916+0000","last_clean":"2024-03-16T17:58:51.454916+0000","last_became_active":"2024-03-16T17:58:49.430168+0000","last_became_peered":"2024-03-16T17:58:49.430168+0000","last_unstale":"2024-03-16T17:58:51.454916+0000","last_undegraded":"2024-03-16T17:58:51.454916+0000","last_fullsized":"2024-03-16T17:58:51.454916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:47:18.543862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566513+0000","last_change":"2024-03-16T17:58:50.469214+0000","last_active":"2024-03-16T17:58:51.566513+0000","last_peered":"2024-03-16T17:58:51.566513+0000","last_clean":"2024-03-16T17:58:51.566513+0000","last_became_active":"2024-03-16T17:58:50.467172+0000","last_became_peered":"2024-03-16T17:58:50.467172+0000","last_unstale":"2024-03-16T17:58:51.566513+0000","last_undegraded":"2024-03-16T17:58:51.566513+0000","last_fullsized":"2024-03-16T17:58:51.566513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:03:24.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071393+0000","last_change":"2024-03-16T17:58:49.440525+0000","last_active":"2024-03-16T17:58:52.071393+0000","last_peered":"2024-03-16T17:58:52.071393+0000","last_clean":"2024-03-16T17:58:52.071393+0000","last_became_active":"2024-03-16T17:58:49.440023+0000","last_became_peered":"2024-03-16T17:58:49.440023+0000","last_unstale":"2024-03-16T17:58:52.071393+0000","last_undegraded":"2024-03-16T17:58:52.071393+0000","last_fullsized":"2024-03-16T17:58:52.071393+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:32:25.953531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070983+0000","last_change":"2024-03-16T17:58:50.470586+0000","last_active":"2024-03-16T17:58:52.070983+0000","last_peered":"2024-03-16T17:58:52.070983+0000","last_clean":"2024-03-16T17:58:52.070983+0000","last_became_active":"2024-03-16T17:58:50.470246+0000","last_became_peered":"2024-03-16T17:58:50.470246+0000","last_unstale":"2024-03-16T17:58:52.070983+0000","last_undegraded":"2024-03-16T17:58:52.070983+0000","last_fullsized":"2024-03-16T17:58:52.070983+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:03:24.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566595+0000","last_change":"2024-03-16T17:58:49.443184+0000","last_active":"2024-03-16T17:58:51.566595+0000","last_peered":"2024-03-16T17:58:51.566595+0000","last_clean":"2024-03-16T17:58:51.566595+0000","last_became_active":"2024-03-16T17:58:49.442622+0000","last_became_peered":"2024-03-16T17:58:49.442622+0000","last_unstale":"2024-03-16T17:58:51.566595+0000","last_undegraded":"2024-03-16T17:58:51.566595+0000","last_fullsized":"2024-03-16T17:58:51.566595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:35:49.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069578+0000","last_change":"2024-03-16T17:58:50.461790+0000","last_active":"2024-03-16T17:58:52.069578+0000","last_peered":"2024-03-16T17:58:52.069578+0000","last_clean":"2024-03-16T17:58:52.069578+0000","last_became_active":"2024-03-16T17:58:50.461409+0000","last_became_peered":"2024-03-16T17:58:50.461409+0000","last_unstale":"2024-03-16T17:58:52.069578+0000","last_undegraded":"2024-03-16T17:58:52.069578+0000","last_fullsized":"2024-03-16T17:58:52.069578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:32:26.966954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.457008+0000","last_change":"2024-03-16T17:58:49.439635+0000","last_active":"2024-03-16T17:58:51.457008+0000","last_peered":"2024-03-16T17:58:51.457008+0000","last_clean":"2024-03-16T17:58:51.457008+0000","last_became_active":"2024-03-16T17:58:49.439105+0000","last_became_peered":"2024-03-16T17:58:49.439105+0000","last_unstale":"2024-03-16T17:58:51.457008+0000","last_undegraded":"2024-03-16T17:58:51.457008+0000","last_fullsized":"2024-03-16T17:58:51.457008+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:54:13.667841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566467+0000","last_change":"2024-03-16T17:58:50.451477+0000","last_active":"2024-03-16T17:58:51.566467+0000","last_peered":"2024-03-16T17:58:51.566467+0000","last_clean":"2024-03-16T17:58:51.566467+0000","last_became_active":"2024-03-16T17:58:50.450768+0000","last_became_peered":"2024-03-16T17:58:50.450768+0000","last_unstale":"2024-03-16T17:58:51.566467+0000","last_undegraded":"2024-03-16T17:58:51.566467+0000","last_fullsized":"2024-03-16T17:58:51.566467+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:19:18.993584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069514+0000","last_change":"2024-03-16T17:58:49.446921+0000","last_active":"2024-03-16T17:58:52.069514+0000","last_peered":"2024-03-16T17:58:52.069514+0000","last_clean":"2024-03-16T17:58:52.069514+0000","last_became_active":"2024-03-16T17:58:49.445870+0000","last_became_peered":"2024-03-16T17:58:49.445870+0000","last_unstale":"2024-03-16T17:58:52.069514+0000","last_undegraded":"2024-03-16T17:58:52.069514+0000","last_fullsized":"2024-03-16T17:58:52.069514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:45:31.631750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070897+0000","last_change":"2024-03-16T17:58:50.473098+0000","last_active":"2024-03-16T17:58:52.070897+0000","last_peered":"2024-03-16T17:58:52.070897+0000","last_clean":"2024-03-16T17:58:52.070897+0000","last_became_active":"2024-03-16T17:58:50.472756+0000","last_became_peered":"2024-03-16T17:58:50.472756+0000","last_unstale":"2024-03-16T17:58:52.070897+0000","last_undegraded":"2024-03-16T17:58:52.070897+0000","last_fullsized":"2024-03-16T17:58:52.070897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:11:34.158762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.565759+0000","last_change":"2024-03-16T17:58:49.440799+0000","last_active":"2024-03-16T17:58:51.565759+0000","last_peered":"2024-03-16T17:58:51.565759+0000","last_clean":"2024-03-16T17:58:51.565759+0000","last_became_active":"2024-03-16T17:58:49.440432+0000","last_became_peered":"2024-03-16T17:58:49.440432+0000","last_unstale":"2024-03-16T17:58:51.565759+0000","last_undegraded":"2024-03-16T17:58:51.565759+0000","last_fullsized":"2024-03-16T17:58:51.565759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:53:50.195359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.072023+0000","last_change":"2024-03-16T17:58:50.459395+0000","last_active":"2024-03-16T17:58:52.072023+0000","last_peered":"2024-03-16T17:58:52.072023+0000","last_clean":"2024-03-16T17:58:52.072023+0000","last_became_active":"2024-03-16T17:58:50.459086+0000","last_became_peered":"2024-03-16T17:58:50.459086+0000","last_unstale":"2024-03-16T17:58:52.072023+0000","last_undegraded":"2024-03-16T17:58:52.072023+0000","last_fullsized":"2024-03-16T17:58:52.072023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:53:51.208782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.566083+0000","last_change":"2024-03-16T17:58:49.443784+0000","last_active":"2024-03-16T17:58:51.566083+0000","last_peered":"2024-03-16T17:58:51.566083+0000","last_clean":"2024-03-16T17:58:51.566083+0000","last_became_active":"2024-03-16T17:58:49.442961+0000","last_became_peered":"2024-03-16T17:58:49.442961+0000","last_unstale":"2024-03-16T17:58:51.566083+0000","last_undegraded":"2024-03-16T17:58:51.566083+0000","last_fullsized":"2024-03-16T17:58:51.566083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:23:46.724279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.457396+0000","last_change":"2024-03-16T17:58:50.454631+0000","last_active":"2024-03-16T17:58:51.457396+0000","last_peered":"2024-03-16T17:58:51.457396+0000","last_clean":"2024-03-16T17:58:51.457396+0000","last_became_active":"2024-03-16T17:58:50.454233+0000","last_became_peered":"2024-03-16T17:58:50.454233+0000","last_unstale":"2024-03-16T17:58:51.457396+0000","last_undegraded":"2024-03-16T17:58:51.457396+0000","last_fullsized":"2024-03-16T17:58:51.457396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:50:54.165035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070698+0000","last_change":"2024-03-16T17:58:49.441265+0000","last_active":"2024-03-16T17:58:52.070698+0000","last_peered":"2024-03-16T17:58:52.070698+0000","last_clean":"2024-03-16T17:58:52.070698+0000","last_became_active":"2024-03-16T17:58:49.440840+0000","last_became_peered":"2024-03-16T17:58:49.440840+0000","last_unstale":"2024-03-16T17:58:52.070698+0000","last_undegraded":"2024-03-16T17:58:52.070698+0000","last_fullsized":"2024-03-16T17:58:52.070698+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:53:50.195359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.072875+0000","last_change":"2024-03-16T17:58:50.454384+0000","last_active":"2024-03-16T17:58:52.072875+0000","last_peered":"2024-03-16T17:58:52.072875+0000","last_clean":"2024-03-16T17:58:52.072875+0000","last_became_active":"2024-03-16T17:58:50.454075+0000","last_became_peered":"2024-03-16T17:58:50.454075+0000","last_unstale":"2024-03-16T17:58:52.072875+0000","last_undegraded":"2024-03-16T17:58:52.072875+0000","last_fullsized":"2024-03-16T17:58:52.072875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:06:02.239701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.457615+0000","last_change":"2024-03-16T17:58:49.441789+0000","last_active":"2024-03-16T17:58:51.457615+0000","last_peered":"2024-03-16T17:58:51.457615+0000","last_clean":"2024-03-16T17:58:51.457615+0000","last_became_active":"2024-03-16T17:58:49.441384+0000","last_became_peered":"2024-03-16T17:58:49.441384+0000","last_unstale":"2024-03-16T17:58:51.457615+0000","last_undegraded":"2024-03-16T17:58:51.457615+0000","last_fullsized":"2024-03-16T17:58:51.457615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:41:19.921507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070654+0000","last_change":"2024-03-16T17:58:50.464530+0000","last_active":"2024-03-16T17:58:52.070654+0000","last_peered":"2024-03-16T17:58:52.070654+0000","last_clean":"2024-03-16T17:58:52.070654+0000","last_became_active":"2024-03-16T17:58:50.464194+0000","last_became_peered":"2024-03-16T17:58:50.464194+0000","last_unstale":"2024-03-16T17:58:52.070654+0000","last_undegraded":"2024-03-16T17:58:52.070654+0000","last_fullsized":"2024-03-16T17:58:52.070654+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:00:53.339915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.457109+0000","last_change":"2024-03-16T17:58:49.442172+0000","last_active":"2024-03-16T17:58:51.457109+0000","last_peered":"2024-03-16T17:58:51.457109+0000","last_clean":"2024-03-16T17:58:51.457109+0000","last_became_active":"2024-03-16T17:58:49.441855+0000","last_became_peered":"2024-03-16T17:58:49.441855+0000","last_unstale":"2024-03-16T17:58:51.457109+0000","last_undegraded":"2024-03-16T17:58:51.457109+0000","last_fullsized":"2024-03-16T17:58:51.457109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:04:38.166526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455870+0000","last_change":"2024-03-16T17:58:50.473632+0000","last_active":"2024-03-16T17:58:51.455870+0000","last_peered":"2024-03-16T17:58:51.455870+0000","last_clean":"2024-03-16T17:58:51.455870+0000","last_became_active":"2024-03-16T17:58:50.473362+0000","last_became_peered":"2024-03-16T17:58:50.473362+0000","last_unstale":"2024-03-16T17:58:51.455870+0000","last_undegraded":"2024-03-16T17:58:51.455870+0000","last_fullsized":"2024-03-16T17:58:51.455870+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:12:35.449102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456953+0000","last_change":"2024-03-16T17:58:49.439277+0000","last_active":"2024-03-16T17:58:51.456953+0000","last_peered":"2024-03-16T17:58:51.456953+0000","last_clean":"2024-03-16T17:58:51.456953+0000","last_became_active":"2024-03-16T17:58:49.438940+0000","last_became_peered":"2024-03-16T17:58:49.438940+0000","last_unstale":"2024-03-16T17:58:51.456953+0000","last_undegraded":"2024-03-16T17:58:51.456953+0000","last_fullsized":"2024-03-16T17:58:51.456953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:15:55.081410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455429+0000","last_change":"2024-03-16T17:58:50.469634+0000","last_active":"2024-03-16T17:58:51.455429+0000","last_peered":"2024-03-16T17:58:51.455429+0000","last_clean":"2024-03-16T17:58:51.455429+0000","last_became_active":"2024-03-16T17:58:50.469368+0000","last_became_peered":"2024-03-16T17:58:50.469368+0000","last_unstale":"2024-03-16T17:58:51.455429+0000","last_undegraded":"2024-03-16T17:58:51.455429+0000","last_fullsized":"2024-03-16T17:58:51.455429+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:41:20.934931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454434+0000","last_change":"2024-03-16T17:58:49.441933+0000","last_active":"2024-03-16T17:58:51.454434+0000","last_peered":"2024-03-16T17:58:51.454434+0000","last_clean":"2024-03-16T17:58:51.454434+0000","last_became_active":"2024-03-16T17:58:49.440983+0000","last_became_peered":"2024-03-16T17:58:49.440983+0000","last_unstale":"2024-03-16T17:58:51.454434+0000","last_undegraded":"2024-03-16T17:58:51.454434+0000","last_fullsized":"2024-03-16T17:58:51.454434+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:07:33.971257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455789+0000","last_change":"2024-03-16T17:58:50.464437+0000","last_active":"2024-03-16T17:58:51.455789+0000","last_peered":"2024-03-16T17:58:51.455789+0000","last_clean":"2024-03-16T17:58:51.455789+0000","last_became_active":"2024-03-16T17:58:50.464102+0000","last_became_peered":"2024-03-16T17:58:50.464102+0000","last_unstale":"2024-03-16T17:58:51.455789+0000","last_undegraded":"2024-03-16T17:58:51.455789+0000","last_fullsized":"2024-03-16T17:58:51.455789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:03:23.747692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071152+0000","last_change":"2024-03-16T17:58:49.445460+0000","last_active":"2024-03-16T17:58:52.071152+0000","last_peered":"2024-03-16T17:58:52.071152+0000","last_clean":"2024-03-16T17:58:52.071152+0000","last_became_active":"2024-03-16T17:58:49.445153+0000","last_became_peered":"2024-03-16T17:58:49.445153+0000","last_unstale":"2024-03-16T17:58:52.071152+0000","last_undegraded":"2024-03-16T17:58:52.071152+0000","last_fullsized":"2024-03-16T17:58:52.071152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:45:31.631750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454839+0000","last_change":"2024-03-16T17:58:50.464874+0000","last_active":"2024-03-16T17:58:51.454839+0000","last_peered":"2024-03-16T17:58:51.454839+0000","last_clean":"2024-03-16T17:58:51.454839+0000","last_became_active":"2024-03-16T17:58:50.463674+0000","last_became_peered":"2024-03-16T17:58:50.463674+0000","last_unstale":"2024-03-16T17:58:51.454839+0000","last_undegraded":"2024-03-16T17:58:51.454839+0000","last_fullsized":"2024-03-16T17:58:51.454839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:43:49.643472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455703+0000","last_change":"2024-03-16T17:58:49.442390+0000","last_active":"2024-03-16T17:58:51.455703+0000","last_peered":"2024-03-16T17:58:51.455703+0000","last_clean":"2024-03-16T17:58:51.455703+0000","last_became_active":"2024-03-16T17:58:49.442119+0000","last_became_peered":"2024-03-16T17:58:49.442119+0000","last_unstale":"2024-03-16T17:58:51.455703+0000","last_undegraded":"2024-03-16T17:58:51.455703+0000","last_fullsized":"2024-03-16T17:58:51.455703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:59:14.480003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.073504+0000","last_change":"2024-03-16T17:58:50.465615+0000","last_active":"2024-03-16T17:58:52.073504+0000","last_peered":"2024-03-16T17:58:52.073504+0000","last_clean":"2024-03-16T17:58:52.073504+0000","last_became_active":"2024-03-16T17:58:50.465291+0000","last_became_peered":"2024-03-16T17:58:50.465291+0000","last_unstale":"2024-03-16T17:58:52.073504+0000","last_undegraded":"2024-03-16T17:58:52.073504+0000","last_fullsized":"2024-03-16T17:58:52.073504+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:14:57.330119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069398+0000","last_change":"2024-03-16T17:58:49.449370+0000","last_active":"2024-03-16T17:58:52.069398+0000","last_peered":"2024-03-16T17:58:52.069398+0000","last_clean":"2024-03-16T17:58:52.069398+0000","last_became_active":"2024-03-16T17:58:49.449033+0000","last_became_peered":"2024-03-16T17:58:49.449033+0000","last_unstale":"2024-03-16T17:58:52.069398+0000","last_undegraded":"2024-03-16T17:58:52.069398+0000","last_fullsized":"2024-03-16T17:58:52.069398+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:54:26.411849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456102+0000","last_change":"2024-03-16T17:58:50.466710+0000","last_active":"2024-03-16T17:58:51.456102+0000","last_peered":"2024-03-16T17:58:51.456102+0000","last_clean":"2024-03-16T17:58:51.456102+0000","last_became_active":"2024-03-16T17:58:50.466424+0000","last_became_peered":"2024-03-16T17:58:50.466424+0000","last_unstale":"2024-03-16T17:58:51.456102+0000","last_undegraded":"2024-03-16T17:58:51.456102+0000","last_fullsized":"2024-03-16T17:58:51.456102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:38:02.893836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070865+0000","last_change":"2024-03-16T17:58:49.441544+0000","last_active":"2024-03-16T17:58:52.070865+0000","last_peered":"2024-03-16T17:58:52.070865+0000","last_clean":"2024-03-16T17:58:52.070865+0000","last_became_active":"2024-03-16T17:58:49.440907+0000","last_became_peered":"2024-03-16T17:58:49.440907+0000","last_unstale":"2024-03-16T17:58:52.070865+0000","last_undegraded":"2024-03-16T17:58:52.070865+0000","last_fullsized":"2024-03-16T17:58:52.070865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:35:49.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.073296+0000","last_change":"2024-03-16T17:58:50.470549+0000","last_active":"2024-03-16T17:58:52.073296+0000","last_peered":"2024-03-16T17:58:52.073296+0000","last_clean":"2024-03-16T17:58:52.073296+0000","last_became_active":"2024-03-16T17:58:50.470269+0000","last_became_peered":"2024-03-16T17:58:50.470269+0000","last_unstale":"2024-03-16T17:58:52.073296+0000","last_undegraded":"2024-03-16T17:58:52.073296+0000","last_fullsized":"2024-03-16T17:58:52.073296+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:18:17.534434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455038+0000","last_change":"2024-03-16T17:58:49.437446+0000","last_active":"2024-03-16T17:58:51.455038+0000","last_peered":"2024-03-16T17:58:51.455038+0000","last_clean":"2024-03-16T17:58:51.455038+0000","last_became_active":"2024-03-16T17:58:49.437170+0000","last_became_peered":"2024-03-16T17:58:49.437170+0000","last_unstale":"2024-03-16T17:58:51.455038+0000","last_undegraded":"2024-03-16T17:58:51.455038+0000","last_fullsized":"2024-03-16T17:58:51.455038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:45:31.631750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454765+0000","last_change":"2024-03-16T17:58:50.462471+0000","last_active":"2024-03-16T17:58:51.454765+0000","last_peered":"2024-03-16T17:58:51.454765+0000","last_clean":"2024-03-16T17:58:51.454765+0000","last_became_active":"2024-03-16T17:58:50.462123+0000","last_became_peered":"2024-03-16T17:58:50.462123+0000","last_unstale":"2024-03-16T17:58:51.454765+0000","last_undegraded":"2024-03-16T17:58:51.454765+0000","last_fullsized":"2024-03-16T17:58:51.454765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:28:15.828722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070535+0000","last_change":"2024-03-16T17:58:49.441844+0000","last_active":"2024-03-16T17:58:52.070535+0000","last_peered":"2024-03-16T17:58:52.070535+0000","last_clean":"2024-03-16T17:58:52.070535+0000","last_became_active":"2024-03-16T17:58:49.441548+0000","last_became_peered":"2024-03-16T17:58:49.441548+0000","last_unstale":"2024-03-16T17:58:52.070535+0000","last_undegraded":"2024-03-16T17:58:52.070535+0000","last_fullsized":"2024-03-16T17:58:52.070535+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:13:38.431719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071010+0000","last_change":"2024-03-16T17:58:50.451937+0000","last_active":"2024-03-16T17:58:52.071010+0000","last_peered":"2024-03-16T17:58:52.071010+0000","last_clean":"2024-03-16T17:58:52.071010+0000","last_became_active":"2024-03-16T17:58:50.451623+0000","last_became_peered":"2024-03-16T17:58:50.451623+0000","last_unstale":"2024-03-16T17:58:52.071010+0000","last_undegraded":"2024-03-16T17:58:52.071010+0000","last_fullsized":"2024-03-16T17:58:52.071010+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:21:37.094237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454745+0000","last_change":"2024-03-16T17:58:49.436289+0000","last_active":"2024-03-16T17:58:51.454745+0000","last_peered":"2024-03-16T17:58:51.454745+0000","last_clean":"2024-03-16T17:58:51.454745+0000","last_became_active":"2024-03-16T17:58:49.435849+0000","last_became_peered":"2024-03-16T17:58:49.435849+0000","last_unstale":"2024-03-16T17:58:51.454745+0000","last_undegraded":"2024-03-16T17:58:51.454745+0000","last_fullsized":"2024-03-16T17:58:51.454745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:13:38.431719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.454613+0000","last_change":"2024-03-16T17:58:50.454567+0000","last_active":"2024-03-16T17:58:51.454613+0000","last_peered":"2024-03-16T17:58:51.454613+0000","last_clean":"2024-03-16T17:58:51.454613+0000","last_became_active":"2024-03-16T17:58:50.454220+0000","last_became_peered":"2024-03-16T17:58:50.454220+0000","last_unstale":"2024-03-16T17:58:51.454613+0000","last_undegraded":"2024-03-16T17:58:51.454613+0000","last_fullsized":"2024-03-16T17:58:51.454613+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:53:51.208782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.069816+0000","last_change":"2024-03-16T17:58:49.445120+0000","last_active":"2024-03-16T17:58:52.069816+0000","last_peered":"2024-03-16T17:58:52.069816+0000","last_clean":"2024-03-16T17:58:52.069816+0000","last_became_active":"2024-03-16T17:58:49.444850+0000","last_became_peered":"2024-03-16T17:58:49.444850+0000","last_unstale":"2024-03-16T17:58:52.069816+0000","last_undegraded":"2024-03-16T17:58:52.069816+0000","last_fullsized":"2024-03-16T17:58:52.069816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:07:33.971257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.565858+0000","last_change":"2024-03-16T17:58:50.448159+0000","last_active":"2024-03-16T17:58:51.565858+0000","last_peered":"2024-03-16T17:58:51.565858+0000","last_clean":"2024-03-16T17:58:51.565858+0000","last_became_active":"2024-03-16T17:58:50.447843+0000","last_became_peered":"2024-03-16T17:58:50.447843+0000","last_unstale":"2024-03-16T17:58:51.565858+0000","last_undegraded":"2024-03-16T17:58:51.565858+0000","last_fullsized":"2024-03-16T17:58:51.565858+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:34:41.770525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.456085+0000","last_change":"2024-03-16T17:58:49.441292+0000","last_active":"2024-03-16T17:58:51.456085+0000","last_peered":"2024-03-16T17:58:51.456085+0000","last_clean":"2024-03-16T17:58:51.456085+0000","last_became_active":"2024-03-16T17:58:49.440973+0000","last_became_peered":"2024-03-16T17:58:49.440973+0000","last_unstale":"2024-03-16T17:58:51.456085+0000","last_undegraded":"2024-03-16T17:58:51.456085+0000","last_fullsized":"2024-03-16T17:58:51.456085+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:38:13.871717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.455472+0000","last_change":"2024-03-16T17:58:50.448187+0000","last_active":"2024-03-16T17:58:51.455472+0000","last_peered":"2024-03-16T17:58:51.455472+0000","last_clean":"2024-03-16T17:58:51.455472+0000","last_became_active":"2024-03-16T17:58:50.447828+0000","last_became_peered":"2024-03-16T17:58:50.447828+0000","last_unstale":"2024-03-16T17:58:51.455472+0000","last_undegraded":"2024-03-16T17:58:51.455472+0000","last_fullsized":"2024-03-16T17:58:51.455472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:46:44.667447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:51.565906+0000","last_change":"2024-03-16T17:58:49.444926+0000","last_active":"2024-03-16T17:58:51.565906+0000","last_peered":"2024-03-16T17:58:51.565906+0000","last_clean":"2024-03-16T17:58:51.565906+0000","last_became_active":"2024-03-16T17:58:49.444648+0000","last_became_peered":"2024-03-16T17:58:49.444648+0000","last_unstale":"2024-03-16T17:58:51.565906+0000","last_undegraded":"2024-03-16T17:58:51.565906+0000","last_fullsized":"2024-03-16T17:58:51.565906+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:21:36.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.074091+0000","last_change":"2024-03-16T17:58:50.468967+0000","last_active":"2024-03-16T17:58:52.074091+0000","last_peered":"2024-03-16T17:58:52.074091+0000","last_clean":"2024-03-16T17:58:52.074091+0000","last_became_active":"2024-03-16T17:58:50.468485+0000","last_became_peered":"2024-03-16T17:58:50.468485+0000","last_unstale":"2024-03-16T17:58:52.074091+0000","last_undegraded":"2024-03-16T17:58:52.074091+0000","last_fullsized":"2024-03-16T17:58:52.074091+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:49.399056+0000","last_clean_scrub_stamp":"2024-03-16T17:58:49.399056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:03:24.289671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.070457+0000","last_change":"2024-03-16T17:58:49.444380+0000","last_active":"2024-03-16T17:58:52.070457+0000","last_peered":"2024-03-16T17:58:52.070457+0000","last_clean":"2024-03-16T17:58:52.070457+0000","last_became_active":"2024-03-16T17:58:49.444051+0000","last_became_peered":"2024-03-16T17:58:49.444051+0000","last_unstale":"2024-03-16T17:58:52.070457+0000","last_undegraded":"2024-03-16T17:58:52.070457+0000","last_fullsized":"2024-03-16T17:58:52.070457+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:58:48.385633+0000","last_clean_scrub_stamp":"2024-03-16T17:58:48.385633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:43:55.619208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-16T17:58:52.071241+0000","last_change":"2024-03-16T17:57:35.560172+0000","last_active":"2024-03-16T17:58:52.071241+0000","last_peered":"2024-03-16T17:58:52.071241+0000","last_clean":"2024-03-16T17:58:52.071241+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T17:58:52.071241+0000","last_undegraded":"2024-03-16T17:58:52.071241+0000","last_fullsized":"2024-03-16T17:58:52.071241+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_clean_scrub_stamp":"2024-03-16T17:57:34.174437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:20:03.039404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":740465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542164,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":150111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":151641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":150077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":151641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542164,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":740375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T17:58:55.402 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-16T17:58:55.402 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-16T17:58:55.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.659+0000 7f3aa8a00640 1 Processor -- start 2024-03-16T17:58:55.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.659+0000 7f3aa8a00640 1 -- start start 2024-03-16T17:58:55.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.661+0000 7f3aa8a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3aa4159960 0x7f3aa406e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.661+0000 7f3aa8a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3aa40cca20 0x7f3aa40cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.661+0000 7f3aa8a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa415c370 con 0x7f3aa406e770 2024-03-16T17:58:55.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.661+0000 7f3aa8a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa4159d40 con 0x7f3aa40cca20 2024-03-16T17:58:55.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.661+0000 7f3aa8a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa406ec20 con 0x7f3aa4159960 2024-03-16T17:58:55.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.661+0000 7f3aa2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3aa40cca20 0x7f3aa40cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:55.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.661+0000 7f3a99400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3aa406e770 0x7f3aa40cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47794/0 (socket says 172.21.15.82:47794) 2024-03-16T17:58:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.661+0000 7f3aa2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3aa40cca20 0x7f3aa40cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37950/0 (socket says 172.21.15.82:37950) 2024-03-16T17:58:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.661+0000 7f3a99400640 1 -- 172.21.15.82:0/2830209805 learned_addr learned my addr 172.21.15.82:0/2830209805 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.661+0000 7f3aa1e00640 1 --2- 172.21.15.82:0/2830209805 >> v2:172.21.15.90:3300/0 conn(0x7f3aa4159960 0x7f3aa406e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.662+0000 7f3aa2800640 1 -- 172.21.15.82:0/2830209805 >> v2:172.21.15.90:3300/0 conn(0x7f3aa4159960 msgr2=0x7f3aa406e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.662+0000 7f3aa2800640 1 --2- 172.21.15.82:0/2830209805 >> v2:172.21.15.90:3300/0 conn(0x7f3aa4159960 0x7f3aa406e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.662+0000 7f3aa2800640 1 -- 172.21.15.82:0/2830209805 >> v1:172.21.15.82:6789/0 conn(0x7f3aa406e770 legacy=0x7f3aa40cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.662+0000 7f3aa2800640 1 -- 172.21.15.82:0/2830209805 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa415c650 con 0x7f3aa40cca20 2024-03-16T17:58:55.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.662+0000 7f3aa1e00640 1 --2- 172.21.15.82:0/2830209805 >> v2:172.21.15.90:3300/0 conn(0x7f3aa4159960 0x7f3aa406e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:55.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.663+0000 7f3aa2800640 1 --2- 172.21.15.82:0/2830209805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 0x7f3aa40cd540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40cc260 tx=0x7f3a9c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166c879504c0c8b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.663+0000 7f3aa1400640 1 -- 172.21.15.82:0/2830209805 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c02cbe0 con 0x7f3aa40cca20 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.663+0000 7f3aa1400640 1 -- 172.21.15.82:0/2830209805 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c02cd40 con 0x7f3aa40cca20 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.663+0000 7f3aa1400640 1 -- 172.21.15.82:0/2830209805 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c034660 con 0x7f3aa40cca20 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.664+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2830209805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 msgr2=0x7f3aa40cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.664+0000 7f3aa8a00640 1 --2- 172.21.15.82:0/2830209805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 0x7f3aa40cd540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40cc260 tx=0x7f3a9c02be30 comp rx=0 tx=0).stop 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.664+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2830209805 shutdown_connections 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.664+0000 7f3aa8a00640 1 --2- 172.21.15.82:0/2830209805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 0x7f3aa40cd540 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.664+0000 7f3aa8a00640 1 --2- 172.21.15.82:0/2830209805 >> v2:172.21.15.90:3300/0 conn(0x7f3aa4159960 0x7f3aa406e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.664+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2830209805 >> 172.21.15.82:0/2830209805 conn(0x7f3aa40b1640 msgr2=0x7f3aa40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.664+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2830209805 shutdown_connections 2024-03-16T17:58:55.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.664+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2830209805 wait complete. 2024-03-16T17:58:55.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.666+0000 7f3aa8a00640 1 Processor -- start 2024-03-16T17:58:55.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.666+0000 7f3aa8a00640 1 -- start start 2024-03-16T17:58:55.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.667+0000 7f3aa8a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa406e770 0x7f3aa40ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.667+0000 7f3aa8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 0x7f3aa41f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.667+0000 7f3aa8a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3aa4159960 0x7f3aa415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.667+0000 7f3aa8a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa415cff0 con 0x7f3aa40cca20 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.667+0000 7f3aa8a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa4158e20 con 0x7f3aa406e770 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.667+0000 7f3aa8a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40b36b0 con 0x7f3aa4159960 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.668+0000 7f3a99400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 0x7f3aa41f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.668+0000 7f3a99400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 0x7f3aa41f67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37962/0 (socket says 172.21.15.82:37962) 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.668+0000 7f3a99400640 1 -- 172.21.15.82:0/2213011837 learned_addr learned my addr 172.21.15.82:0/2213011837 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.668+0000 7f3aa1e00640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa406e770 0x7f3aa40ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.668+0000 7f3a99400640 1 -- 172.21.15.82:0/2213011837 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3aa4159960 msgr2=0x7f3aa415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.668+0000 7f3a99400640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3aa4159960 0x7f3aa415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.668+0000 7f3a99400640 1 -- 172.21.15.82:0/2213011837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa406e770 msgr2=0x7f3aa40ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.668+0000 7f3a99400640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa406e770 0x7f3aa40ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.668+0000 7f3a99400640 1 -- 172.21.15.82:0/2213011837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c009a70 con 0x7f3aa40cca20 2024-03-16T17:58:55.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.668+0000 7f3a99400640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 0x7f3aa41f67a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00c430 tx=0x7f3a8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:55.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.669+0000 7f3a9aa00640 1 -- 172.21.15.82:0/2213011837 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c005ce0 con 0x7f3aa40cca20 2024-03-16T17:58:55.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.669+0000 7f3a9aa00640 1 -- 172.21.15.82:0/2213011837 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c005e40 con 0x7f3aa40cca20 2024-03-16T17:58:55.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.669+0000 7f3a9aa00640 1 -- 172.21.15.82:0/2213011837 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0072f0 con 0x7f3aa40cca20 2024-03-16T17:58:55.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.669+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2213011837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa40caa60 con 0x7f3aa40cca20 2024-03-16T17:58:55.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.669+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2213011837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa40b2d80 con 0x7f3aa40cca20 2024-03-16T17:58:55.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.670+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2213011837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa4067f30 con 0x7f3aa40cca20 2024-03-16T17:58:55.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.671+0000 7f3a9aa00640 1 -- 172.21.15.82:0/2213011837 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f3a8c007510 con 0x7f3aa40cca20 2024-03-16T17:58:55.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.672+0000 7f3a9aa00640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3a8006b3b0 0x7f3a8006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:55.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.673+0000 7f3a9aa00640 1 -- 172.21.15.82:0/2213011837 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a8c0892c0 con 0x7f3aa40cca20 2024-03-16T17:58:55.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.673+0000 7f3aa1e00640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3a8006b3b0 0x7f3a8006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:55.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.673+0000 7f3aa1e00640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3a8006b3b0 0x7f3a8006d870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400b840 tx=0x7f3a94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:55.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.677+0000 7f3a9aa00640 1 -- 172.21.15.82:0/2213011837 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3a8c053e80 con 0x7f3aa40cca20 2024-03-16T17:58:55.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.942+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2213011837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3aa40ceb80 con 0x7f3aa40cca20 2024-03-16T17:58:55.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.946+0000 7f3a9aa00640 1 -- 172.21.15.82:0/2213011837 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f3a8c057540 con 0x7f3aa40cca20 2024-03-16T17:58:55.950 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-03-16T17:58:55.950 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBY3fVlVabyEhAAXRXAiI2UCfBSDg0N+ezAyw== 2024-03-16T17:58:55.950 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-03-16T17:58:55.950 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-03-16T17:58:55.950 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-16T17:58:55.950 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-03-16T17:58:55.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2213011837 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3a8006b3b0 msgr2=0x7f3a8006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3a8006b3b0 0x7f3a8006d870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400b840 tx=0x7f3a94008040 comp rx=0 tx=0).stop 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2213011837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 msgr2=0x7f3aa41f67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 0x7f3aa41f67a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c00c430 tx=0x7f3a8c00c8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2213011837 shutdown_connections 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f3a8006b3b0 0x7f3a8006d870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f3aa4159960 0x7f3aa415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa40cca20 0x7f3aa41f67a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 --2- 172.21.15.82:0/2213011837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3aa406e770 0x7f3aa40ca520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2213011837 >> 172.21.15.82:0/2213011837 conn(0x7f3aa40b1640 msgr2=0x7f3aa40cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2213011837 shutdown_connections 2024-03-16T17:58:55.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:55.949+0000 7f3aa8a00640 1 -- 172.21.15.82:0/2213011837 wait complete. 2024-03-16T17:58:55.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T17:58:56.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.163+0000 7f8ff6200640 1 Processor -- start 2024-03-16T17:58:56.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.163+0000 7f8ff6200640 1 -- start start 2024-03-16T17:58:56.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.164+0000 7f8ff6200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ff01454d0 0x7f8ff00b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:56.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.164+0000 7f8ff6200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8ff00cdf80 0x7f8ff00cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:56.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.164+0000 7f8ff6200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff00b4b90 con 0x7f8ff00cfe70 2024-03-16T17:58:56.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.165+0000 7f8ff6200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff00d0320 con 0x7f8ff00cdf80 2024-03-16T17:58:56.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.165+0000 7f8ff6200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00cf500 con 0x7f8ff01454d0 2024-03-16T17:58:56.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.165+0000 7f8fef400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ff01454d0 0x7f8ff00b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:56.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.165+0000 7f8fe7e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8ff00cdf80 0x7f8ff00cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:56.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.165+0000 7f8fe7e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8ff00cdf80 0x7f8ff00cf930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:52136/0 (socket says 172.21.15.82:52136) 2024-03-16T17:58:56.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.165+0000 7f8fe7e00640 1 -- 172.21.15.82:0/2434030927 learned_addr learned my addr 172.21.15.82:0/2434030927 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:56.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.166+0000 7f8fe7e00640 1 -- 172.21.15.82:0/2434030927 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ff01454d0 msgr2=0x7f8ff00b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:56.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.166+0000 7f8fe7e00640 1 --2- 172.21.15.82:0/2434030927 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ff01454d0 0x7f8ff00b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:56.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.166+0000 7f8fe7e00640 1 -- 172.21.15.82:0/2434030927 >> v1:172.21.15.90:6789/0 conn(0x7f8ff00cfe70 legacy=0x7f8ff015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:56.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.166+0000 7f8fe7e00640 1 -- 172.21.15.82:0/2434030927 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff01458b0 con 0x7f8ff00cdf80 2024-03-16T17:58:56.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.167+0000 7f8fe7e00640 1 --2- 172.21.15.82:0/2434030927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ff00cdf80 0x7f8ff00cf930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009870 tx=0x7f8fd802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca5ec1cdfc93cf9e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:56.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8feea00640 1 -- 172.21.15.82:0/2434030927 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8030840 con 0x7f8ff00cdf80 2024-03-16T17:58:56.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8feea00640 1 -- 172.21.15.82:0/2434030927 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd80309a0 con 0x7f8ff00cdf80 2024-03-16T17:58:56.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8feea00640 1 -- 172.21.15.82:0/2434030927 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8030d30 con 0x7f8ff00cdf80 2024-03-16T17:58:56.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8ff6200640 1 -- 172.21.15.82:0/2434030927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ff00cdf80 msgr2=0x7f8ff00cf930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:56.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8ff6200640 1 --2- 172.21.15.82:0/2434030927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ff00cdf80 0x7f8ff00cf930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009870 tx=0x7f8fd802b980 comp rx=0 tx=0).stop 2024-03-16T17:58:56.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8ff6200640 1 -- 172.21.15.82:0/2434030927 shutdown_connections 2024-03-16T17:58:56.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8ff6200640 1 --2- 172.21.15.82:0/2434030927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ff00cdf80 0x7f8ff00cf930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:56.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8ff6200640 1 --2- 172.21.15.82:0/2434030927 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ff01454d0 0x7f8ff00b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:56.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8ff6200640 1 -- 172.21.15.82:0/2434030927 >> 172.21.15.82:0/2434030927 conn(0x7f8ff006ae70 msgr2=0x7f8ff006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:56.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8ff6200640 1 -- 172.21.15.82:0/2434030927 shutdown_connections 2024-03-16T17:58:56.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.168+0000 7f8ff6200640 1 -- 172.21.15.82:0/2434030927 wait complete. 2024-03-16T17:58:56.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.170+0000 7f8ff6200640 1 Processor -- start 2024-03-16T17:58:56.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.170+0000 7f8ff6200640 1 -- start start 2024-03-16T17:58:56.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.171+0000 7f8ff6200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff00cdf80 0x7f8ff00b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:56.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.171+0000 7f8ff6200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ff00cfe70 0x7f8ff00af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:56.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.171+0000 7f8ff6200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ff01454d0 0x7f8ff0140790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.171+0000 7f8ff6200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff0158600 con 0x7f8ff00cdf80 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.171+0000 7f8ff6200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00b07a0 con 0x7f8ff01454d0 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.171+0000 7f8ff6200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00b01e0 con 0x7f8ff00cfe70 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.171+0000 7f8fef400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff00cdf80 0x7f8ff00b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.171+0000 7f8fef400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff00cdf80 0x7f8ff00b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37970/0 (socket says 172.21.15.82:37970) 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.171+0000 7f8fef400640 1 -- 172.21.15.82:0/3257414564 learned_addr learned my addr 172.21.15.82:0/3257414564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fefe00640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ff01454d0 0x7f8ff0140790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fe7e00640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ff00cfe70 0x7f8ff00af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fef400640 1 -- 172.21.15.82:0/3257414564 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ff00cfe70 msgr2=0x7f8ff00af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fef400640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ff00cfe70 0x7f8ff00af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fef400640 1 -- 172.21.15.82:0/3257414564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ff01454d0 msgr2=0x7f8ff0140790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fef400640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ff01454d0 0x7f8ff0140790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fef400640 1 -- 172.21.15.82:0/3257414564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd80094d0 con 0x7f8ff00cdf80 2024-03-16T17:58:56.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fef400640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff00cdf80 0x7f8ff00b1a80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e870 tx=0x7f8fe000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:56.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fe7400640 1 -- 172.21.15.82:0/3257414564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00094f0 con 0x7f8ff00cdf80 2024-03-16T17:58:56.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fe7400640 1 -- 172.21.15.82:0/3257414564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe000c940 con 0x7f8ff00cdf80 2024-03-16T17:58:56.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.172+0000 7f8fe7400640 1 -- 172.21.15.82:0/3257414564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe000ccf0 con 0x7f8ff00cdf80 2024-03-16T17:58:56.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.173+0000 7f8ff6200640 1 -- 172.21.15.82:0/3257414564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff00b04c0 con 0x7f8ff00cdf80 2024-03-16T17:58:56.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.173+0000 7f8ff6200640 1 -- 172.21.15.82:0/3257414564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff0133900 con 0x7f8ff00cdf80 2024-03-16T17:58:56.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.174+0000 7f8ff6200640 1 -- 172.21.15.82:0/3257414564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff00b3d10 con 0x7f8ff00cdf80 2024-03-16T17:58:56.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.175+0000 7f8fe7400640 1 -- 172.21.15.82:0/3257414564 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8fe0013470 con 0x7f8ff00cdf80 2024-03-16T17:58:56.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.175+0000 7f8fe7400640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8fcc06b3b0 0x7f8fcc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:56.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.176+0000 7f8fe7400640 1 -- 172.21.15.82:0/3257414564 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fe0089d70 con 0x7f8ff00cdf80 2024-03-16T17:58:56.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.176+0000 7f8fe7e00640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8fcc06b3b0 0x7f8fcc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:56.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.176+0000 7f8fe7e00640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8fcc06b3b0 0x7f8fcc06d870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8036000 tx=0x7f8fd8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:56.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.180+0000 7f8fe7400640 1 -- 172.21.15.82:0/3257414564 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8fe00549c0 con 0x7f8ff00cdf80 2024-03-16T17:58:56.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.397+0000 7f8ff6200640 1 -- 172.21.15.82:0/3257414564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ff0067f30 con 0x7f8ff00cdf80 2024-03-16T17:58:56.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.398+0000 7f8fe7400640 1 -- 172.21.15.82:0/3257414564 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4974 (secure 0 0 0) 0x7f8fe0017090 con 0x7f8ff00cdf80 2024-03-16T17:58:56.402 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:58:56.402 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5100,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3801030113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3801030113},{"type":"v1","addr":"172.21.15.82:6836","nonce":3801030113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2266371611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2266371611},{"type":"v1","addr":"172.21.15.82:6837","nonce":2266371611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/693422260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":693422260},{"type":"v1","addr":"172.21.15.90:6834","nonce":693422260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:58:49.876364+0000","modified":"2024-03-16T17:58:52.961483+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6835/2421794786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":2421794786},{"type":"v1","addr":"172.21.15.90:6835","nonce":2421794786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5118,"qdb_cluster":[5118]},"id":2}]} 2024-03-16T17:58:56.406 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 16 2024-03-16T17:58:56.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 -- 172.21.15.82:0/3257414564 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8fcc06b3b0 msgr2=0x7f8fcc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:56.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8fcc06b3b0 0x7f8fcc06d870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8036000 tx=0x7f8fd8008030 comp rx=0 tx=0).stop 2024-03-16T17:58:56.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 -- 172.21.15.82:0/3257414564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff00cdf80 msgr2=0x7f8ff00b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:56.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff00cdf80 0x7f8ff00b1a80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e870 tx=0x7f8fe000ed30 comp rx=0 tx=0).stop 2024-03-16T17:58:56.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 -- 172.21.15.82:0/3257414564 shutdown_connections 2024-03-16T17:58:56.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8fcc06b3b0 0x7f8fcc06d870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:56.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ff01454d0 0x7f8ff0140790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:56.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ff00cfe70 0x7f8ff00af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:56.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 --2- 172.21.15.82:0/3257414564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ff00cdf80 0x7f8ff00b1a80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:56.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 -- 172.21.15.82:0/3257414564 >> 172.21.15.82:0/3257414564 conn(0x7f8ff006ae70 msgr2=0x7f8ff0070980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:56.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 -- 172.21.15.82:0/3257414564 shutdown_connections 2024-03-16T17:58:56.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:56.401+0000 7f8ff6200640 1 -- 172.21.15.82:0/3257414564 wait complete. 2024-03-16T17:58:56.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-16T17:58:49.876364+0000', 'modified': '2024-03-16T17:58:52.961483+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5118}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5118': {'gid': 5118, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.90:6835/2421794786', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.90:6833', 'nonce': 2421794786}, {'type': 'v1', 'addr': '172.21.15.90:6835', 'nonce': 2421794786}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5118, 'qdb_cluster': [5118]} 2024-03-16T17:58:56.826 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-16T17:58:56.826 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-16T17:58:57.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.050+0000 7fa8ab400640 1 Processor -- start 2024-03-16T17:58:57.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.051+0000 7fa8ab400640 1 -- start start 2024-03-16T17:58:57.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.052+0000 7fa8ab400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8a40ce190 0x7fa8a415b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.052+0000 7fa8ab400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fa8a4070fb0 0x7fa8a4071420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.052+0000 7fa8ab400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a40b3520 con 0x7fa8a40cf390 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.052+0000 7fa8ab400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a406c870 con 0x7fa8a40ce190 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.052+0000 7fa8ab400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a415b980 con 0x7fa8a4070fb0 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.052+0000 7fa8a8a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8a40ce190 0x7fa8a415b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.052+0000 7fa89be00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa8a40cf390 0x7fa8a40ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47810/0 (socket says 172.21.15.82:47810) 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.052+0000 7fa8a8a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8a40ce190 0x7fa8a415b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37984/0 (socket says 172.21.15.82:37984) 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.052+0000 7fa89be00640 1 -- 172.21.15.82:0/2584123917 learned_addr learned my addr 172.21.15.82:0/2584123917 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.053+0000 7fa8a9400640 1 --2- 172.21.15.82:0/2584123917 >> v2:172.21.15.90:3300/0 conn(0x7fa8a4070fb0 0x7fa8a4071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.053+0000 7fa8a8a00640 1 -- 172.21.15.82:0/2584123917 >> v2:172.21.15.90:3300/0 conn(0x7fa8a4070fb0 msgr2=0x7fa8a4071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.053+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/2584123917 >> v2:172.21.15.90:3300/0 conn(0x7fa8a4070fb0 0x7fa8a4071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.053+0000 7fa8a8a00640 1 -- 172.21.15.82:0/2584123917 >> v1:172.21.15.82:6789/0 conn(0x7fa8a40cf390 legacy=0x7fa8a40ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.053+0000 7fa8a8a00640 1 -- 172.21.15.82:0/2584123917 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a40b3ae0 con 0x7fa8a40ce190 2024-03-16T17:58:57.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.053+0000 7fa8a3e00640 1 -- 172.21.15.82:0/2584123917 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1311729905 0 0) 0x7fa8a40b3520 con 0x7fa8a40cf390 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.054+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/2584123917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 0x7fa8a415b440 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa8a406ea30 tx=0x7fa89402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f484f4cebb4929 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.054+0000 7fa8a3e00640 1 -- 172.21.15.82:0/2584123917 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894031830 con 0x7fa8a40ce190 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.054+0000 7fa8a3e00640 1 -- 172.21.15.82:0/2584123917 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa894031990 con 0x7fa8a40ce190 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.054+0000 7fa8a3e00640 1 -- 172.21.15.82:0/2584123917 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894031d20 con 0x7fa8a40ce190 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.055+0000 7fa8ab400640 1 -- 172.21.15.82:0/2584123917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 msgr2=0x7fa8a415b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.055+0000 7fa8ab400640 1 --2- 172.21.15.82:0/2584123917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 0x7fa8a415b440 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa8a406ea30 tx=0x7fa89402b8d0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.055+0000 7fa8ab400640 1 -- 172.21.15.82:0/2584123917 shutdown_connections 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.055+0000 7fa8ab400640 1 --2- 172.21.15.82:0/2584123917 >> v2:172.21.15.90:3300/0 conn(0x7fa8a4070fb0 0x7fa8a4071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.055+0000 7fa8ab400640 1 --2- 172.21.15.82:0/2584123917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 0x7fa8a415b440 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.055+0000 7fa8ab400640 1 -- 172.21.15.82:0/2584123917 >> 172.21.15.82:0/2584123917 conn(0x7fa8a40b17b0 msgr2=0x7fa8a40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.055+0000 7fa8ab400640 1 -- 172.21.15.82:0/2584123917 shutdown_connections 2024-03-16T17:58:57.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.055+0000 7fa8ab400640 1 -- 172.21.15.82:0/2584123917 wait complete. 2024-03-16T17:58:57.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.057+0000 7fa8ab400640 1 Processor -- start 2024-03-16T17:58:57.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.057+0000 7fa8ab400640 1 -- start start 2024-03-16T17:58:57.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.058+0000 7fa8ab400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8a4070fb0 0x7fa8a4138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.058+0000 7fa8ab400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 0x7fa8a415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.058+0000 7fa8ab400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8a40cf390 0x7fa8a40cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa8ab400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a4139500 con 0x7fa8a40ce190 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa8ab400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a41596b0 con 0x7fa8a4070fb0 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa8ab400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a4159990 con 0x7fa8a40cf390 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa89be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 0x7fa8a415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa89be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 0x7fa8a415ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37996/0 (socket says 172.21.15.82:37996) 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa89be00640 1 -- 172.21.15.82:0/3355650377 learned_addr learned my addr 172.21.15.82:0/3355650377 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8a4070fb0 0x7fa8a4138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa8a9400640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8a40cf390 0x7fa8a40cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa89be00640 1 -- 172.21.15.82:0/3355650377 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8a40cf390 msgr2=0x7fa8a40cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa89be00640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8a40cf390 0x7fa8a40cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa89be00640 1 -- 172.21.15.82:0/3355650377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8a4070fb0 msgr2=0x7fa8a4138fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa89be00640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8a4070fb0 0x7fa8a4138fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa89be00640 1 -- 172.21.15.82:0/3355650377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa894009400 con 0x7fa8a40ce190 2024-03-16T17:58:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa8a9400640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8a40cf390 0x7fa8a40cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.059+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8a4070fb0 0x7fa8a4138fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.060+0000 7fa89be00640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 0x7fa8a415ab80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00c430 tx=0x7fa88c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.060+0000 7fa8a1600640 1 -- 172.21.15.82:0/3355650377 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0074e0 con 0x7fa8a40ce190 2024-03-16T17:58:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.060+0000 7fa8a1600640 1 -- 172.21.15.82:0/3355650377 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c007660 con 0x7fa8a40ce190 2024-03-16T17:58:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.060+0000 7fa8a1600640 1 -- 172.21.15.82:0/3355650377 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c004a20 con 0x7fa8a40ce190 2024-03-16T17:58:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.060+0000 7fa8ab400640 1 -- 172.21.15.82:0/3355650377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a4159c70 con 0x7fa8a40ce190 2024-03-16T17:58:57.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.060+0000 7fa8ab400640 1 -- 172.21.15.82:0/3355650377 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a415a230 con 0x7fa8a40ce190 2024-03-16T17:58:57.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.062+0000 7fa8a1600640 1 -- 172.21.15.82:0/3355650377 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fa88c004b80 con 0x7fa8a40ce190 2024-03-16T17:58:57.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.063+0000 7fa8a1600640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa88006b3b0 0x7fa88006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.063+0000 7fa8a1600640 1 -- 172.21.15.82:0/3355650377 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa88c089a90 con 0x7fa8a40ce190 2024-03-16T17:58:57.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.063+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa88006b3b0 0x7fa88006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.064+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa88006b3b0 0x7fa88006d870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa894036000 tx=0x7fa894008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:57.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.064+0000 7fa8ae1b8740 1 Processor -- start 2024-03-16T17:58:57.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.064+0000 7fa8ae1b8740 1 -- start start 2024-03-16T17:58:57.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.065+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1f630 0x56323a9f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.065+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1fa10 0x56323aa223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.065+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323a9f2b10 0x56323aa22900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.065+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56323a966b30 con 0x56323aa1fa10 2024-03-16T17:58:57.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.065+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56323a964060 con 0x56323a9f2b10 2024-03-16T17:58:57.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.065+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56323a95f840 con 0x56323aa1f630 2024-03-16T17:58:57.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.065+0000 7fa89be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1fa10 0x56323aa223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.065+0000 7fa89be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1fa10 0x56323aa223c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38008/0 (socket says 172.21.15.82:38008) 2024-03-16T17:58:57.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.065+0000 7fa89be00640 1 -- 172.21.15.82:0/1765127213 learned_addr learned my addr 172.21.15.82:0/1765127213 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:57.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/1765127213 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1f630 0x56323a9f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa8a9400640 1 --2- 172.21.15.82:0/1765127213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323a9f2b10 0x56323aa22900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa89be00640 1 -- 172.21.15.82:0/1765127213 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1f630 msgr2=0x56323a9f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa89be00640 1 --2- 172.21.15.82:0/1765127213 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1f630 0x56323a9f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa89be00640 1 -- 172.21.15.82:0/1765127213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323a9f2b10 msgr2=0x56323aa22900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa89be00640 1 --2- 172.21.15.82:0/1765127213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323a9f2b10 0x56323aa22900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa89be00640 1 -- 172.21.15.82:0/1765127213 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56323a9274c0 con 0x56323aa1fa10 2024-03-16T17:58:57.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa89be00640 1 --2- 172.21.15.82:0/1765127213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1fa10 0x56323aa223c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0548a0 tx=0x7fa88c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b51e59a8e444fe server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa899200640 1 -- 172.21.15.82:0/1765127213 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c058440 con 0x56323aa1fa10 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa899200640 1 -- 172.21.15.82:0/1765127213 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c05c590 con 0x56323aa1fa10 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.066+0000 7fa899200640 1 -- 172.21.15.82:0/1765127213 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c060b50 con 0x56323aa1fa10 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.067+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/1765127213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1fa10 msgr2=0x56323aa223c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.067+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/1765127213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1fa10 0x56323aa223c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0548a0 tx=0x7fa88c00f000 comp rx=0 tx=0).stop 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.067+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/1765127213 shutdown_connections 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.067+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/1765127213 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323a9f2b10 0x56323aa22900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.067+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/1765127213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1fa10 0x56323aa223c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.067+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/1765127213 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1f630 0x56323a9f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.067+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/1765127213 >> 172.21.15.82:0/1765127213 conn(0x56323a9f4840 msgr2=0x56323aa3d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.067+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/1765127213 shutdown_connections 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.067+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/1765127213 wait complete. 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.068+0000 7fa8ae1b8740 1 -- start start 2024-03-16T17:58:57.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.068+0000 7fa8ae1b8740 1 Processor -- start 2024-03-16T17:58:57.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.069+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1f630 0x56323aa25690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.069+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1fa10 0x56323aa285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.069+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa0ead0 0x56323aa28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.069+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56323a904ea0 con 0x56323aa1f630 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.069+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56323a904b40 con 0x56323aa1fa10 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.069+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56323a6a7f20 con 0x56323aa0ead0 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.069+0000 7fa8ae1b8740 10 client.0 _ll_register_callbacks cb 0x56323a918950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.069+0000 7fa8a8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1f630 0x56323aa25690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa8a8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1f630 0x56323aa25690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38016/0 (socket says 172.21.15.82:38016) 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa8a8a00640 1 -- 172.21.15.82:0/439909436 learned_addr learned my addr 172.21.15.82:0/439909436 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa89be00640 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1fa10 0x56323aa285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa8a9400640 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa0ead0 0x56323aa28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa8a8a00640 1 -- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa0ead0 msgr2=0x56323aa28b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa0ead0 0x56323aa28b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa8a8a00640 1 -- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1fa10 msgr2=0x56323aa285c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1fa10 0x56323aa285c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa8a8a00640 1 -- 172.21.15.82:0/439909436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a415a230 con 0x56323aa1f630 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1f630 0x56323aa25690 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa894006ce0 tx=0x7fa894004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8042872f1e6a6fb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.070+0000 7fa891600640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa891600640 1 -- 172.21.15.82:0/439909436 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89404c070 con 0x56323aa1f630 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa891600640 1 -- 172.21.15.82:0/439909436 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa894045e40 con 0x56323aa1f630 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/439909436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56323a983c40 con 0x56323aa1f630 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa891600640 1 -- 172.21.15.82:0/439909436 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa89403f070 con 0x56323aa1f630 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa8ae1b8740 10 client.5286 fetch_fsmap learned FSMap version 16 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/439909436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x56323a95dec0 con 0x56323aa1f630 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa891600640 1 -- 172.21.15.82:0/439909436 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56323a983c40 con 0x56323aa1f630 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa891600640 1 -- 172.21.15.82:0/439909436 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 16) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fa894046630 con 0x56323aa1f630 2024-03-16T17:58:57.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa8ae1b8740 10 client.5286 fetch_fsmap finished waiting for FSMap version 16 2024-03-16T17:58:57.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa8ae1b8740 10 client.5286 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T17:58:57.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa8ae1b8740 20 client.5286 populate_metadata read hostname 'smithi082' 2024-03-16T17:58:57.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:6833/2421794786,v1:172.21.15.90:6835/2421794786] conn(0x56323aa58630 0x56323aa5aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa8ae1b8740 4 client.5286 mds_command: new command op to 5118 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T17:58:57.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.071+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/439909436 --> [v2:172.21.15.90:6833/2421794786,v1:172.21.15.90:6835/2421794786] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56323a65f6c0 con 0x56323aa58630 2024-03-16T17:58:57.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.072+0000 7fa89be00640 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:6833/2421794786,v1:172.21.15.90:6835/2421794786] conn(0x56323aa58630 0x56323aa5aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.073+0000 7fa89be00640 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:6833/2421794786,v1:172.21.15.90:6835/2421794786] conn(0x56323aa58630 0x56323aa5aa20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0d33bb934b4ab1 server_cookie=98c5f020731a7325 in_seq=0 out_seq=0 2024-03-16T17:58:57.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.073+0000 7fa891600640 10 client.5286 ms_handle_connect on v2:172.21.15.90:6833/2421794786 2024-03-16T17:58:57.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa891600640 1 -- 172.21.15.82:0/439909436 <== mds.0 v2:172.21.15.90:6833/2421794786 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56323a65f6c0 con 0x56323aa58630 2024-03-16T17:58:57.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa891600640 10 client.5286 handle_command_reply: tid=0 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa8ae1b8740 1 client.5286 shutdown 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/439909436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1f630 msgr2=0x56323aa25690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1f630 0x56323aa25690 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa894006ce0 tx=0x7fa894004730 comp rx=0 tx=0).stop 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/439909436 shutdown_connections 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:6833/2421794786,v1:172.21.15.90:6835/2421794786] conn(0x56323aa58630 0x56323aa5aa20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa0ead0 0x56323aa28b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1fa10 0x56323aa285c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/439909436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa1f630 0x56323aa25690 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa891600640 0 client.5286 ms_handle_reset on v2:172.21.15.90:6833/2421794786 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/439909436 >> 172.21.15.82:0/439909436 conn(0x56323a9f2b10 msgr2=0x56323a9f9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.075+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/439909436 shutdown_connections 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.076+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/439909436 wait complete. 2024-03-16T17:58:57.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.076+0000 7fa8ae1b8740 20 client.5286 trim_cache size 0 max 16384 2024-03-16T17:58:57.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.093+0000 7fa8ae1b8740 1 Processor -- start 2024-03-16T17:58:57.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.094+0000 7fa8ae1b8740 1 -- start start 2024-03-16T17:58:57.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.094+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323a9f1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.094+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1f630 0x56323aa26cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.094+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1fa10 0x56323aa27230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.094+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56323aa1d470 con 0x56323aa0ead0 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.094+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56323a9f30c0 con 0x56323aa1fa10 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.094+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56323aa4fd40 con 0x56323aa1f630 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.095+0000 7fa8a8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323a9f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.095+0000 7fa8a8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323a9f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38022/0 (socket says 172.21.15.82:38022) 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.095+0000 7fa8a8a00640 1 -- 172.21.15.82:0/2013817610 learned_addr learned my addr 172.21.15.82:0/2013817610 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.095+0000 7fa8a8a00640 1 -- 172.21.15.82:0/2013817610 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1f630 msgr2=0x56323aa26cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.095+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/2013817610 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1f630 0x56323aa26cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.095+0000 7fa8a8a00640 1 -- 172.21.15.82:0/2013817610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1fa10 msgr2=0x56323aa27230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.095+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/2013817610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1fa10 0x56323aa27230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.095+0000 7fa8a8a00640 1 -- 172.21.15.82:0/2013817610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56323a9f21a0 con 0x56323aa0ead0 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.095+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/2013817610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323a9f1580 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa894046b40 tx=0x7fa89404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdba767942c1e7c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa893e00640 1 -- 172.21.15.82:0/2013817610 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894045410 con 0x56323aa0ead0 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa893e00640 1 -- 172.21.15.82:0/2013817610 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa894045e40 con 0x56323aa0ead0 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa893e00640 1 -- 172.21.15.82:0/2013817610 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894046630 con 0x56323aa0ead0 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/2013817610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 msgr2=0x56323a9f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/2013817610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323a9f1580 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa894046b40 tx=0x7fa89404c000 comp rx=0 tx=0).stop 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/2013817610 shutdown_connections 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/2013817610 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1fa10 0x56323aa27230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/2013817610 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1f630 0x56323aa26cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/2013817610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323a9f1580 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/2013817610 >> 172.21.15.82:0/2013817610 conn(0x56323a9f2b10 msgr2=0x56323a9f92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/2013817610 shutdown_connections 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.096+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/2013817610 wait complete. 2024-03-16T17:58:57.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.097+0000 7fa8ae1b8740 1 -- start start 2024-03-16T17:58:57.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.097+0000 7fa8ae1b8740 1 Processor -- start 2024-03-16T17:58:57.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.098+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323aaa5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.098+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1f630 0x56323aa1d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.098+0000 7fa8ae1b8740 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1fa10 0x56323aa288a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.098+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56323aa28170 con 0x56323aa0ead0 2024-03-16T17:58:57.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.098+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x56323aa28410 con 0x56323aa1f630 2024-03-16T17:58:57.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.098+0000 7fa8ae1b8740 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x56323aaac340 con 0x56323aa1fa10 2024-03-16T17:58:57.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.098+0000 7fa8ae1b8740 10 client.0 _ll_register_callbacks cb 0x7fa8940455c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T17:58:57.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.098+0000 7fa8a8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323aaa5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa8a8a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323aaa5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38036/0 (socket says 172.21.15.82:38036) 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa8a8a00640 1 -- 172.21.15.82:0/3296110652 learned_addr learned my addr 172.21.15.82:0/3296110652 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa89be00640 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1f630 0x56323aa1d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa8a8a00640 1 -- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1fa10 msgr2=0x56323aa288a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa8a9400640 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1fa10 0x56323aa288a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1fa10 0x56323aa288a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa8a8a00640 1 -- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1f630 msgr2=0x56323aa1d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1f630 0x56323aa1d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa8a8a00640 1 -- 172.21.15.82:0/3296110652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56323a95dec0 con 0x56323aa0ead0 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa89be00640 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1f630 0x56323aa1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa8a8a00640 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323aaa5390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa89402bee0 tx=0x7fa894009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c604d35fb59525e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa891600640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.099+0000 7fa891600640 1 -- 172.21.15.82:0/3296110652 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89404c040 con 0x56323aa0ead0 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa891600640 1 -- 172.21.15.82:0/3296110652 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa894003610 con 0x56323aa0ead0 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/3296110652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa88c05c590 con 0x56323aa0ead0 2024-03-16T17:58:57.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa891600640 1 -- 172.21.15.82:0/3296110652 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa894045410 con 0x56323aa0ead0 2024-03-16T17:58:57.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa8ae1b8740 10 client.5292 fetch_fsmap learned FSMap version 16 2024-03-16T17:58:57.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/3296110652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x56323a8fa8b0 con 0x56323aa0ead0 2024-03-16T17:58:57.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa891600640 1 -- 172.21.15.82:0/3296110652 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c05c590 con 0x56323aa0ead0 2024-03-16T17:58:57.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa891600640 1 -- 172.21.15.82:0/3296110652 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 16) v1 ==== 3314+0+0 (secure 0 0 0) 0x7fa8940037d0 con 0x56323aa0ead0 2024-03-16T17:58:57.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa8ae1b8740 10 client.5292 fetch_fsmap finished waiting for FSMap version 16 2024-03-16T17:58:57.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa8ae1b8740 10 client.5292 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T17:58:57.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa8ae1b8740 20 client.5292 populate_metadata read hostname 'smithi082' 2024-03-16T17:58:57.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:6833/2421794786,v1:172.21.15.90:6835/2421794786] conn(0x56323aa5ea40 0x56323aa5ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa8ae1b8740 4 client.5292 mds_command: new command op to 5118 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T17:58:57.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.100+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/3296110652 --> [v2:172.21.15.90:6833/2421794786,v1:172.21.15.90:6835/2421794786] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56323a6b0270 con 0x56323aa5ea40 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.101+0000 7fa89be00640 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:6833/2421794786,v1:172.21.15.90:6835/2421794786] conn(0x56323aa5ea40 0x56323aa5ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.102+0000 7fa89be00640 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:6833/2421794786,v1:172.21.15.90:6835/2421794786] conn(0x56323aa5ea40 0x56323aa5ee20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e373c417fce56ac server_cookie=6b762ca48902cb86 in_seq=0 out_seq=0 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.102+0000 7fa891600640 10 client.5292 ms_handle_connect on v2:172.21.15.90:6833/2421794786 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.102+0000 7fa891600640 1 -- 172.21.15.82:0/3296110652 <== mds.0 v2:172.21.15.90:6833/2421794786 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56323a6b0270 con 0x56323aa5ea40 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.102+0000 7fa891600640 10 client.5292 handle_command_reply: tid=0 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.102+0000 7fa8ae1b8740 1 client.5292 shutdown 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/3296110652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 msgr2=0x56323aaa5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323aaa5390 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa89402bee0 tx=0x7fa894009190 comp rx=0 tx=0).stop 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/3296110652 shutdown_connections 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:6833/2421794786,v1:172.21.15.90:6835/2421794786] conn(0x56323aa5ea40 0x56323aa5ee20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x56323aa1fa10 0x56323aa288a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56323aa1f630 0x56323aa1d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 1 --2- 172.21.15.82:0/3296110652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56323aa0ead0 0x56323aaa5390 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa891600640 0 client.5292 ms_handle_reset on v2:172.21.15.90:6833/2421794786 2024-03-16T17:58:57.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/3296110652 >> 172.21.15.82:0/3296110652 conn(0x56323a9f2b10 msgr2=0x56323a9f1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:57.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/3296110652 shutdown_connections 2024-03-16T17:58:57.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 1 -- 172.21.15.82:0/3296110652 wait complete. 2024-03-16T17:58:57.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.103+0000 7fa8ae1b8740 20 client.5292 trim_cache size 0 max 16384 2024-03-16T17:58:57.109 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T17:58:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "08082c2e-06a4-4a16-ab2e-9e9e244a6661", 2024-03-16T17:58:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-03-16T17:58:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 5118, 2024-03-16T17:58:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-03-16T17:58:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-03-16T17:58:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-03-16T17:58:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 6.6770561920000002, 2024-03-16T17:58:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 16, 2024-03-16T17:58:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 34, 2024-03-16T17:58:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 34, 2024-03-16T17:58:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 15.490009052 2024-03-16T17:58:57.110 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T17:58:57.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 -- 172.21.15.82:0/3355650377 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa88006b3b0 msgr2=0x7fa88006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa88006b3b0 0x7fa88006d870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa894036000 tx=0x7fa894008030 comp rx=0 tx=0).stop 2024-03-16T17:58:57.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 -- 172.21.15.82:0/3355650377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 msgr2=0x7fa8a415ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 0x7fa8a415ab80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00c430 tx=0x7fa88c00c8f0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 -- 172.21.15.82:0/3355650377 shutdown_connections 2024-03-16T17:58:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fa88006b3b0 0x7fa88006d870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fa8a40cf390 0x7fa8a40cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8a40ce190 0x7fa8a415ab80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 --2- 172.21.15.82:0/3355650377 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa8a4070fb0 0x7fa8a4138fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 -- 172.21.15.82:0/3355650377 >> 172.21.15.82:0/3355650377 conn(0x7fa8a40b17b0 msgr2=0x7fa8a4154220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 -- 172.21.15.82:0/3355650377 shutdown_connections 2024-03-16T17:58:57.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.104+0000 7fa8ab400640 1 -- 172.21.15.82:0/3355650377 wait complete. 2024-03-16T17:58:57.124 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-16T17:58:57.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.323+0000 7fae6e800640 1 Processor -- start 2024-03-16T17:58:57.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.323+0000 7fae6e800640 1 -- start start 2024-03-16T17:58:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.324+0000 7fae6e800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae680b44d0 0x7fae680712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae6e800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae680cf1f0 0x7fae6815df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae6e800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae68070cd0 con 0x7fae6815e460 2024-03-16T17:58:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae6e800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6815e910 con 0x7fae680cf1f0 2024-03-16T17:58:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae6e800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae6806e660 con 0x7fae680b44d0 2024-03-16T17:58:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae5ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae680cf1f0 0x7fae6815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae5ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae680cf1f0 0x7fae6815df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38052/0 (socket says 172.21.15.82:38052) 2024-03-16T17:58:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae5ea00640 1 -- 172.21.15.82:0/3351960171 learned_addr learned my addr 172.21.15.82:0/3351960171 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae67400640 1 --2- 172.21.15.82:0/3351960171 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae680b44d0 0x7fae680712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae5ea00640 1 -- 172.21.15.82:0/3351960171 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae680b44d0 msgr2=0x7fae680712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae5ea00640 1 --2- 172.21.15.82:0/3351960171 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae680b44d0 0x7fae680712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae5ea00640 1 -- 172.21.15.82:0/3351960171 >> v1:172.21.15.90:6789/0 conn(0x7fae6815e460 legacy=0x7fae6815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T17:58:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.325+0000 7fae5ea00640 1 -- 172.21.15.82:0/3351960171 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae680b3ac0 con 0x7fae680cf1f0 2024-03-16T17:58:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.327+0000 7fae5ea00640 1 --2- 172.21.15.82:0/3351960171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680cf1f0 0x7fae6815df20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fae5000bb00 tx=0x7fae5002dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a557455f74d3e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.327+0000 7fae66a00640 1 -- 172.21.15.82:0/3351960171 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5002ec50 con 0x7fae680cf1f0 2024-03-16T17:58:57.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.327+0000 7fae66a00640 1 -- 172.21.15.82:0/3351960171 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae50032070 con 0x7fae680cf1f0 2024-03-16T17:58:57.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.327+0000 7fae66a00640 1 -- 172.21.15.82:0/3351960171 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50038400 con 0x7fae680cf1f0 2024-03-16T17:58:57.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.327+0000 7fae6e800640 1 -- 172.21.15.82:0/3351960171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680cf1f0 msgr2=0x7fae6815df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.327+0000 7fae6e800640 1 --2- 172.21.15.82:0/3351960171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680cf1f0 0x7fae6815df20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fae5000bb00 tx=0x7fae5002dea0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.328+0000 7fae6e800640 1 -- 172.21.15.82:0/3351960171 shutdown_connections 2024-03-16T17:58:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.328+0000 7fae6e800640 1 --2- 172.21.15.82:0/3351960171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680cf1f0 0x7fae6815df20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.328+0000 7fae6e800640 1 --2- 172.21.15.82:0/3351960171 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae680b44d0 0x7fae680712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.328+0000 7fae6e800640 1 -- 172.21.15.82:0/3351960171 >> 172.21.15.82:0/3351960171 conn(0x7fae6806ae70 msgr2=0x7fae6806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.328+0000 7fae6e800640 1 -- 172.21.15.82:0/3351960171 shutdown_connections 2024-03-16T17:58:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.328+0000 7fae6e800640 1 -- 172.21.15.82:0/3351960171 wait complete. 2024-03-16T17:58:57.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.329+0000 7fae6e800640 1 Processor -- start 2024-03-16T17:58:57.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.330+0000 7fae6e800640 1 -- start start 2024-03-16T17:58:57.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.330+0000 7fae6e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680b44d0 0x7fae68156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.330+0000 7fae6e800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae680cf1f0 0x7fae681f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae6e800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae6815e460 0x7fae681510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae6e800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae68157430 con 0x7fae680b44d0 2024-03-16T17:58:57.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae6e800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae681567b0 con 0x7fae680cf1f0 2024-03-16T17:58:57.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae6e800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae68156a90 con 0x7fae6815e460 2024-03-16T17:58:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae67400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680b44d0 0x7fae68156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae67400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680b44d0 0x7fae68156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:38062/0 (socket says 172.21.15.82:38062) 2024-03-16T17:58:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae67400640 1 -- 172.21.15.82:0/1512121448 learned_addr learned my addr 172.21.15.82:0/1512121448 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T17:58:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae67e00640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae6815e460 0x7fae681510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae5ea00640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae680cf1f0 0x7fae681f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae67400640 1 -- 172.21.15.82:0/1512121448 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae6815e460 msgr2=0x7fae681510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae67400640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae6815e460 0x7fae681510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae67400640 1 -- 172.21.15.82:0/1512121448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae680cf1f0 msgr2=0x7fae681f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae67400640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae680cf1f0 0x7fae681f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.331+0000 7fae67400640 1 -- 172.21.15.82:0/1512121448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5000b760 con 0x7fae680b44d0 2024-03-16T17:58:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.332+0000 7fae67400640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680b44d0 0x7fae68156120 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fae5800d800 tx=0x7fae5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.332+0000 7fae5fe00640 1 -- 172.21.15.82:0/1512121448 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5800b6c0 con 0x7fae680b44d0 2024-03-16T17:58:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.332+0000 7fae5fe00640 1 -- 172.21.15.82:0/1512121448 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5800b820 con 0x7fae680b44d0 2024-03-16T17:58:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.332+0000 7fae6e800640 1 -- 172.21.15.82:0/1512121448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae68156d70 con 0x7fae680b44d0 2024-03-16T17:58:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.332+0000 7fae5fe00640 1 -- 172.21.15.82:0/1512121448 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5800bbb0 con 0x7fae680b44d0 2024-03-16T17:58:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.332+0000 7fae6e800640 1 -- 172.21.15.82:0/1512121448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae681515f0 con 0x7fae680b44d0 2024-03-16T17:58:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.333+0000 7fae6e800640 1 -- 172.21.15.82:0/1512121448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae68154e00 con 0x7fae680b44d0 2024-03-16T17:58:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.334+0000 7fae5fe00640 1 -- 172.21.15.82:0/1512121448 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fae5800bd10 con 0x7fae680b44d0 2024-03-16T17:58:57.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.335+0000 7fae5fe00640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fae4406b3b0 0x7fae4406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T17:58:57.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.335+0000 7fae5fe00640 1 -- 172.21.15.82:0/1512121448 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae5808a0e0 con 0x7fae680b44d0 2024-03-16T17:58:57.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.335+0000 7fae5ea00640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fae4406b3b0 0x7fae4406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T17:58:57.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.336+0000 7fae5ea00640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fae4406b3b0 0x7fae4406d870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fae50039000 tx=0x7fae50034000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T17:58:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.339+0000 7fae5fe00640 1 -- 172.21.15.82:0/1512121448 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fae68154e00 con 0x7fae680b44d0 2024-03-16T17:58:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.578+0000 7fae6e800640 1 -- 172.21.15.82:0/1512121448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fae68154e00 con 0x7fae680b44d0 2024-03-16T17:58:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.579+0000 7fae5fe00640 1 -- 172.21.15.82:0/1512121448 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fae68154e00 con 0x7fae680b44d0 2024-03-16T17:58:57.583 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T17:58:57.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.581+0000 7fae6e800640 1 -- 172.21.15.82:0/1512121448 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fae4406b3b0 msgr2=0x7fae4406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.581+0000 7fae6e800640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fae4406b3b0 0x7fae4406d870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fae50039000 tx=0x7fae50034000 comp rx=0 tx=0).stop 2024-03-16T17:58:57.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.581+0000 7fae6e800640 1 -- 172.21.15.82:0/1512121448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680b44d0 msgr2=0x7fae68156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T17:58:57.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.581+0000 7fae6e800640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680b44d0 0x7fae68156120 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fae5800d800 tx=0x7fae5800dcc0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.582+0000 7fae6e800640 1 -- 172.21.15.82:0/1512121448 shutdown_connections 2024-03-16T17:58:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.582+0000 7fae6e800640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fae4406b3b0 0x7fae4406d870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.582+0000 7fae6e800640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fae6815e460 0x7fae681510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.582+0000 7fae6e800640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae680cf1f0 0x7fae681f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.582+0000 7fae6e800640 1 --2- 172.21.15.82:0/1512121448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae680b44d0 0x7fae68156120 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T17:58:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.582+0000 7fae6e800640 1 -- 172.21.15.82:0/1512121448 >> 172.21.15.82:0/1512121448 conn(0x7fae6806ae70 msgr2=0x7fae6806d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T17:58:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.582+0000 7fae6e800640 1 -- 172.21.15.82:0/1512121448 shutdown_connections 2024-03-16T17:58:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T17:58:57.582+0000 7fae6e800640 1 -- 172.21.15.82:0/1512121448 wait complete. 2024-03-16T17:58:57.604 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-16T17:58:57.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T17:58:57.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-03-16T17:58:57.604 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T17:58:57.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T17:58:57.604 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-16T17:58:57.604 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-16T17:58:57.605 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T17:58:57.605 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T17:58:57.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:57.605 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-03-16T17:58:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T17:58:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T17:58:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T17:58:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-03-16T17:58:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:57.654 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-03-16T17:58:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42516sec preferred_lft 42516sec 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether ae:cb:75:75:37:47 brd ff:ff:ff:ff:ff:ff 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-16T17:58:57.655 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::b05f:e0ff:fecd:8301/64 scope link 2024-03-16T17:58:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:57.656 INFO:teuthology.orchestra.run.smithi082.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-16T17:58:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether be:72:9e:97:6e:1f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T17:58:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::bc72:9eff:fe97:6e1f/64 scope link 2024-03-16T17:58:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:57.656 INFO:teuthology.orchestra.run.smithi082.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-16T17:58:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether ae:cb:75:75:37:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-16T17:58:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::accb:75ff:fe75:3747/64 scope link 2024-03-16T17:58:57.656 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-03-16T17:58:57.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:57.657 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T17:58:57.742 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-03-16T17:58:57.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:57.746 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-03-16T17:58:57.799 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-03-16T17:58:57.799 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T17:58:57.800 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-16T17:58:57.800 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-16T17:58:57.800 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T17:58:57.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:57.801 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-03-16T17:58:57.801 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-03-16T17:58:57.801 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-16T17:58:57.801 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-16T17:58:57.801 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-16T17:58:57.801 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-03-16T17:58:57.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:57.983 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-16T17:58:58.033 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-16T17:58:58.033 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-16T17:58:58.033 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T17:58:58.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:58.033 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-03-16T17:58:58.124 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T17:58:58.124 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-16T17:58:58.176 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-16T17:58:58.176 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T17:58:58.176 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-16T17:58:58.228 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-16T17:58:58.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:58.228 DEBUG:teuthology.orchestra.run.smithi082:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T17:58:58.359 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T17:58:58.359 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-16T17:58:58.359 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-03-16T17:58:58.381 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T17:58:58.381 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-16T17:58:58.381 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-03-16T17:58:58.381 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.90:3300,172.21.15.90:3301" 2024-03-16T17:58:58.381 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T17:58:58.406 INFO:teuthology.orchestra.run.smithi082.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-16T17:58:58.407 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-16T17:58:58.407 INFO:teuthology.orchestra.run.smithi082.stdout:invalid new device string format 2024-03-16T17:58:58.407 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:3300,172.21.15.90:3300,172.21.15.90:3301" 2024-03-16T17:58:58.407 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-16T17:58:58.407 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: resolved to: "172.21.15.82:6789,172.21.15.90:6789,172.21.15.90:6790" 2024-03-16T17:58:58.407 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.82:6789,172.21.15.90:6789,172.21.15.90:6790:/ 2024-03-16T17:58:58.407 INFO:teuthology.orchestra.run.smithi082.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=08082c2e-06a4-4a16-ab2e-9e9e244a6661" will pass to kernel 2024-03-16T17:58:58.409 INFO:teuthology.orchestra.run.smithi082.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-16T17:58:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: You just mounted a file system that supports labels which does not 2024-03-16T17:58:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-16T17:58:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: applications will generate AVC messages and not be allowed access to 2024-03-16T17:58:58.409 INFO:teuthology.orchestra.run.smithi082.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-16T17:58:58.415 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-16T17:58:58.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:58.415 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-16T17:58:58.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:58.499 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-16T17:58:58.569 INFO:teuthology.orchestra.run.smithi082.stdout:client5301 2024-03-16T17:58:58.569 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T17:58:58.569 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T17:58:58.623 DEBUG:teuthology.orchestra.run.smithi082:> mktemp -d --suffix=xfstests-dev 2024-03-16T17:58:58.674 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.YEFrEDdeoexfstests-dev 2024-03-16T17:58:58.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:58.674 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.YEFrEDdeoexfstests-dev) 2024-03-16T17:58:58.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:58.773 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.YEFrEDdeoexfstests-dev) 2024-03-16T17:58:58.836 INFO:teuthology.orchestra.run.smithi082.stderr:Cloning into '/tmp/tmp.YEFrEDdeoexfstests-dev'... 2024-03-16T17:58:59.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:59.867 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-03-16T17:58:59.932 DEBUG:teuthology.orchestra.run.smithi082:> mktemp -d --suffix=test 2024-03-16T17:58:59.977 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.8INGWKbrFetest 2024-03-16T17:58:59.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T17:58:59.978 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-03-16T17:59:00.032 DEBUG:teuthology.orchestra.run.smithi082:> mktemp -d --suffix=scratch 2024-03-16T17:59:00.083 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.npBvz5wtGascratch 2024-03-16T17:59:00.083 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-03-16T17:59:00.083 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-03-16T17:59:00.943 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:04:36 ago on Sat 16 Mar 2024 05:54:24 PM UTC. 2024-03-16T17:59:01.046 INFO:teuthology.orchestra.run.smithi082.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-03-16T17:59:01.046 INFO:teuthology.orchestra.run.smithi082.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-03-16T17:59:01.048 INFO:teuthology.orchestra.run.smithi082.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-03-16T17:59:01.049 INFO:teuthology.orchestra.run.smithi082.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-03-16T17:59:01.050 INFO:teuthology.orchestra.run.smithi082.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-03-16T17:59:01.052 INFO:teuthology.orchestra.run.smithi082.stdout:Package e2fsprogs-1.46.5-3.el9.x86_64 is already installed. 2024-03-16T17:59:01.054 INFO:teuthology.orchestra.run.smithi082.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-03-16T17:59:01.055 INFO:teuthology.orchestra.run.smithi082.stdout:Package gcc-11.4.1-2.1.el9.x86_64 is already installed. 2024-03-16T17:59:01.057 INFO:teuthology.orchestra.run.smithi082.stdout:Package git-2.39.3-1.el9.x86_64 is already installed. 2024-03-16T17:59:01.060 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-03-16T17:59:01.061 INFO:teuthology.orchestra.run.smithi082.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-03-16T17:59:01.063 INFO:teuthology.orchestra.run.smithi082.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-03-16T17:59:01.064 INFO:teuthology.orchestra.run.smithi082.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-03-16T17:59:01.065 INFO:teuthology.orchestra.run.smithi082.stdout:Package lvm2-9:2.03.21-3.el9.x86_64 is already installed. 2024-03-16T17:59:01.066 INFO:teuthology.orchestra.run.smithi082.stdout:Package make-1:4.3-7.el9.x86_64 is already installed. 2024-03-16T17:59:01.067 INFO:teuthology.orchestra.run.smithi082.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-03-16T17:59:01.068 INFO:teuthology.orchestra.run.smithi082.stdout:Package python3-3.9.17-2.el9.x86_64 is already installed. 2024-03-16T17:59:01.070 INFO:teuthology.orchestra.run.smithi082.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-03-16T17:59:01.071 INFO:teuthology.orchestra.run.smithi082.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-03-16T17:59:01.074 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-03-16T17:59:01.144 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2024-03-16T17:59:01.147 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-16T17:59:01.147 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2024-03-16T17:59:01.147 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: fio x86_64 3.35-1.el9 CentOS-AppStream 6.2 M 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: kernel-devel x86_64 5.14.0-427.el9 appstream 22 M 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: libcap-devel x86_64 2.48-9.el9 CentOS-AppStream 33 k 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: sqlite x86_64 3.34.1-7.el9 CentOS-AppStream 747 k 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: udftools x86_64 2.2-5.el9 CentOS-AppStream 144 k 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: cpp x86_64 11.4.1-3.el9 appstream 11 M 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: device-mapper x86_64 9:1.02.197-2.el9 baseos 141 k 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: device-mapper-event x86_64 9:1.02.197-2.el9 baseos 34 k 2024-03-16T17:59:01.148 INFO:teuthology.orchestra.run.smithi082.stdout: device-mapper-event-libs x86_64 9:1.02.197-2.el9 baseos 32 k 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: device-mapper-libs x86_64 9:1.02.197-2.el9 baseos 179 k 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: e2fsprogs x86_64 1.46.5-5.el9 baseos 1.0 M 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: e2fsprogs-libs x86_64 1.46.5-5.el9 baseos 222 k 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: gcc x86_64 11.4.1-3.el9 appstream 32 M 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: gcc-c++ x86_64 11.4.1-3.el9 appstream 13 M 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: gcc-plugin-annobin x86_64 11.4.1-3.el9 appstream 47 k 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: git x86_64 2.43.0-1.el9 appstream 51 k 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: git-core x86_64 2.43.0-1.el9 appstream 4.4 M 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: git-core-doc noarch 2.43.0-1.el9 appstream 2.9 M 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err x86_64 1.46.5-5.el9 baseos 26 k 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: libgcc x86_64 11.4.1-3.el9 baseos 92 k 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: libgomp x86_64 11.4.1-3.el9 baseos 268 k 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: libss x86_64 1.46.5-5.el9 baseos 32 k 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: libstdc++ x86_64 11.4.1-3.el9 baseos 743 k 2024-03-16T17:59:01.149 INFO:teuthology.orchestra.run.smithi082.stdout: libstdc++-devel x86_64 11.4.1-3.el9 appstream 2.4 M 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: lvm2 x86_64 9:2.03.23-2.el9 baseos 1.5 M 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: lvm2-libs x86_64 9:2.03.23-2.el9 baseos 1.0 M 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: make x86_64 1:4.3-8.el9 baseos 536 k 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: perl-Git noarch 2.43.0-1.el9 appstream 38 k 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: python-unversioned-command noarch 3.9.18-3.el9 CentOS-AppStream 8.3 k 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3 x86_64 3.9.18-3.el9 CentOS-BaseOS 25 k 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel x86_64 3.9.18-3.el9 CentOS-AppStream 204 k 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: python3-libs x86_64 3.9.18-3.el9 CentOS-BaseOS 7.3 M 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: sqlite-libs x86_64 3.34.1-7.el9 CentOS-BaseOS 619 k 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: rmt x86_64 2:1.6-6.el9 CentOS-BaseOS 45 k 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout:Installing weak dependencies: 2024-03-16T17:59:01.150 INFO:teuthology.orchestra.run.smithi082.stdout: fio-engine-http x86_64 3.35-1.el9 CentOS-AppStream 17 k 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout: fio-engine-libaio x86_64 3.35-1.el9 CentOS-AppStream 14 k 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout: fio-engine-nbd x86_64 3.35-1.el9 CentOS-AppStream 13 k 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout: fio-engine-rados x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout: fio-engine-rbd x86_64 3.35-1.el9 CentOS-AppStream 15 k 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout: fio-engine-rdma x86_64 3.35-1.el9 CentOS-AppStream 20 k 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout:Install 17 Packages 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 28 Packages 2024-03-16T17:59:01.151 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T17:59:01.152 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 109 M 2024-03-16T17:59:01.152 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2024-03-16T17:59:01.987 INFO:teuthology.orchestra.run.smithi082.stdout:(1/45): fio-engine-http-3.35-1.el9.x86_64.rpm 74 kB/s | 17 kB 00:00 2024-03-16T17:59:02.002 INFO:teuthology.orchestra.run.smithi082.stdout:(2/45): fio-engine-libaio-3.35-1.el9.x86_64.rpm 59 kB/s | 14 kB 00:00 2024-03-16T17:59:02.053 INFO:teuthology.orchestra.run.smithi082.stdout:(3/45): fio-engine-nbd-3.35-1.el9.x86_64.rpm 214 kB/s | 13 kB 00:00 2024-03-16T17:59:02.086 INFO:teuthology.orchestra.run.smithi082.stdout:(4/45): fio-engine-rados-3.35-1.el9.x86_64.rpm 179 kB/s | 15 kB 00:00 2024-03-16T17:59:02.212 INFO:teuthology.orchestra.run.smithi082.stdout:(5/45): fio-3.35-1.el9.x86_64.rpm 14 MB/s | 6.2 MB 00:00 2024-03-16T17:59:02.228 INFO:teuthology.orchestra.run.smithi082.stdout:(6/45): fio-engine-rbd-3.35-1.el9.x86_64.rpm 88 kB/s | 15 kB 00:00 2024-03-16T17:59:02.254 INFO:teuthology.orchestra.run.smithi082.stdout:(7/45): fio-engine-rdma-3.35-1.el9.x86_64.rpm 128 kB/s | 20 kB 00:00 2024-03-16T17:59:02.279 INFO:teuthology.orchestra.run.smithi082.stdout:(8/45): libcap-devel-2.48-9.el9.x86_64.rpm 674 kB/s | 33 kB 00:00 2024-03-16T17:59:02.321 INFO:teuthology.orchestra.run.smithi082.stdout:(9/45): sqlite-3.34.1-7.el9.x86_64.rpm 8.1 MB/s | 747 kB 00:00 2024-03-16T17:59:02.354 INFO:teuthology.orchestra.run.smithi082.stdout:(10/45): udftools-2.2-5.el9.x86_64.rpm 1.5 MB/s | 144 kB 00:00 2024-03-16T17:59:02.380 INFO:teuthology.orchestra.run.smithi082.stdout:(11/45): rmt-1.6-6.el9.x86_64.rpm 803 kB/s | 45 kB 00:00 2024-03-16T17:59:02.480 INFO:teuthology.orchestra.run.smithi082.stdout:(12/45): gdbm-1.19-4.el9.x86_64.rpm 1.3 MB/s | 132 kB 00:00 2024-03-16T17:59:02.555 INFO:teuthology.orchestra.run.smithi082.stdout:(13/45): gdbm-devel-1.19-4.el9.x86_64.rpm 789 kB/s | 59 kB 00:00 2024-03-16T17:59:02.739 INFO:teuthology.orchestra.run.smithi082.stdout:(14/45): dump-0.4-0.51.b47.el9.x86_64.rpm 481 kB/s | 88 kB 00:00 2024-03-16T17:59:02.864 INFO:teuthology.orchestra.run.smithi082.stdout:(15/45): indent-2.2.13-5.el9.x86_64.rpm 1.4 MB/s | 179 kB 00:00 2024-03-16T17:59:02.898 INFO:teuthology.orchestra.run.smithi082.stdout:(16/45): python-unversioned-command-3.9.18-3.el 254 kB/s | 8.3 kB 00:00 2024-03-16T17:59:02.940 INFO:teuthology.orchestra.run.smithi082.stdout:(17/45): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 204 kB 00:00 2024-03-16T17:59:02.974 INFO:teuthology.orchestra.run.smithi082.stdout:(18/45): python3-3.9.18-3.el9.x86_64.rpm 769 kB/s | 25 kB 00:00 2024-03-16T17:59:03.174 INFO:teuthology.orchestra.run.smithi082.stdout:(19/45): python3-libs-3.9.18-3.el9.x86_64.rpm 37 MB/s | 7.3 MB 00:00 2024-03-16T17:59:03.233 INFO:teuthology.orchestra.run.smithi082.stdout:(20/45): sqlite-libs-3.34.1-7.el9.x86_64.rpm 11 MB/s | 619 kB 00:00 2024-03-16T17:59:03.341 INFO:teuthology.orchestra.run.smithi082.stdout:(21/45): device-mapper-1.02.197-2.el9.x86_64.rp 1.3 MB/s | 141 kB 00:00 2024-03-16T17:59:03.392 INFO:teuthology.orchestra.run.smithi082.stdout:(22/45): device-mapper-event-1.02.197-2.el9.x86 695 kB/s | 34 kB 00:00 2024-03-16T17:59:03.442 INFO:teuthology.orchestra.run.smithi082.stdout:(23/45): device-mapper-event-libs-1.02.197-2.el 658 kB/s | 32 kB 00:00 2024-03-16T17:59:03.542 INFO:teuthology.orchestra.run.smithi082.stdout:(24/45): device-mapper-libs-1.02.197-2.el9.x86_ 1.8 MB/s | 179 kB 00:00 2024-03-16T17:59:03.593 INFO:teuthology.orchestra.run.smithi082.stdout:(25/45): libnbd-1.18.1-3.el9.x86_64.rpm 135 kB/s | 166 kB 00:01 2024-03-16T17:59:03.710 INFO:teuthology.orchestra.run.smithi082.stdout:(26/45): e2fsprogs-libs-1.46.5-5.el9.x86_64.rpm 1.9 MB/s | 222 kB 00:00 2024-03-16T17:59:03.752 INFO:teuthology.orchestra.run.smithi082.stdout:(27/45): e2fsprogs-1.46.5-5.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-03-16T17:59:03.810 INFO:teuthology.orchestra.run.smithi082.stdout:(28/45): libcom_err-1.46.5-5.el9.x86_64.rpm 266 kB/s | 26 kB 00:00 2024-03-16T17:59:03.852 INFO:teuthology.orchestra.run.smithi082.stdout:(29/45): libgcc-11.4.1-3.el9.x86_64.rpm 930 kB/s | 92 kB 00:00 2024-03-16T17:59:03.927 INFO:teuthology.orchestra.run.smithi082.stdout:(30/45): libgomp-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 268 kB 00:00 2024-03-16T17:59:03.953 INFO:teuthology.orchestra.run.smithi082.stdout:(31/45): libss-1.46.5-5.el9.x86_64.rpm 319 kB/s | 32 kB 00:00 2024-03-16T17:59:04.078 INFO:teuthology.orchestra.run.smithi082.stdout:(32/45): libstdc++-11.4.1-3.el9.x86_64.rpm 4.9 MB/s | 743 kB 00:00 2024-03-16T17:59:04.137 INFO:teuthology.orchestra.run.smithi082.stdout:(33/45): lvm2-2.03.23-2.el9.x86_64.rpm 8.4 MB/s | 1.5 MB 00:00 2024-03-16T17:59:04.237 INFO:teuthology.orchestra.run.smithi082.stdout:(34/45): make-4.3-8.el9.x86_64.rpm 5.3 MB/s | 536 kB 00:00 2024-03-16T17:59:04.287 INFO:teuthology.orchestra.run.smithi082.stdout:(35/45): lvm2-libs-2.03.23-2.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-03-16T17:59:05.656 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:59:05.664 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:59:18.120 INFO:teuthology.orchestra.run.smithi082.stdout:(36/45): cpp-11.4.1-3.el9.x86_64.rpm 784 kB/s | 11 MB 00:13 2024-03-16T17:59:31.043 INFO:teuthology.orchestra.run.smithi082.stdout:(37/45): kernel-devel-5.14.0-427.el9.x86_64.rpm 768 kB/s | 22 MB 00:28 2024-03-16T17:59:31.734 INFO:teuthology.orchestra.run.smithi082.stdout:(38/45): gcc-plugin-annobin-11.4.1-3.el9.x86_64 69 kB/s | 47 kB 00:00 2024-03-16T17:59:32.435 INFO:teuthology.orchestra.run.smithi082.stdout:(39/45): git-2.43.0-1.el9.x86_64.rpm 73 kB/s | 51 kB 00:00 2024-03-16T17:59:35.417 INFO:teuthology.orchestra.run.smithi082.stdout:(40/45): gcc-c++-11.4.1-3.el9.x86_64.rpm 756 kB/s | 13 MB 00:17 2024-03-16T17:59:35.815 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:59:35.823 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T17:59:38.841 INFO:teuthology.orchestra.run.smithi082.stdout:(41/45): git-core-2.43.0-1.el9.x86_64.rpm 708 kB/s | 4.4 MB 00:06 2024-03-16T17:59:39.757 INFO:teuthology.orchestra.run.smithi082.stdout:(42/45): git-core-doc-2.43.0-1.el9.noarch.rpm 690 kB/s | 2.9 MB 00:04 2024-03-16T17:59:40.432 INFO:teuthology.orchestra.run.smithi082.stdout:(43/45): perl-Git-2.43.0-1.el9.noarch.rpm 56 kB/s | 38 kB 00:00 2024-03-16T17:59:42.507 INFO:teuthology.orchestra.run.smithi082.stdout:(44/45): libstdc++-devel-11.4.1-3.el9.x86_64.rp 676 kB/s | 2.4 MB 00:03 2024-03-16T17:59:45.915 INFO:teuthology.orchestra.run.smithi082.stdout:(45/45): gcc-11.4.1-3.el9.x86_64.rpm 793 kB/s | 32 MB 00:41 2024-03-16T17:59:45.920 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2024-03-16T17:59:45.920 INFO:teuthology.orchestra.run.smithi082.stdout:Total 2.4 MB/s | 109 MB 00:44 2024-03-16T17:59:47.669 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2024-03-16T17:59:48.089 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2024-03-16T17:59:48.090 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2024-03-16T17:59:49.707 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2024-03-16T17:59:49.707 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2024-03-16T17:59:52.955 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2024-03-16T17:59:53.046 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libgcc-11.4.1-3.el9.x86_64 1/73 2024-03-16T17:59:53.132 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libgcc-11.4.1-3.el9.x86_64 1/73 2024-03-16T17:59:53.235 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libcom_err-1.46.5-5.el9.x86_64 2/73 2024-03-16T17:59:53.330 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libstdc++-11.4.1-3.el9.x86_64 3/73 2024-03-16T17:59:53.392 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : e2fsprogs-libs-1.46.5-5.el9.x86_64 4/73 2024-03-16T17:59:53.486 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : device-mapper-libs-9:1.02.197-2.el9.x86_64 5/73 2024-03-16T17:59:53.537 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : device-mapper-9:1.02.197-2.el9.x86_64 6/73 2024-03-16T17:59:53.874 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 7/73 2024-03-16T17:59:54.076 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : git-core-2.43.0-1.el9.x86_64 8/73 2024-03-16T17:59:54.145 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : make-1:4.3-8.el9.x86_64 9/73 2024-03-16T17:59:55.711 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : sqlite-libs-3.34.1-7.el9.x86_64 10/73 2024-03-16T17:59:55.852 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : python3-libs-3.9.18-3.el9.x86_64 11/73 2024-03-16T17:59:55.879 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : python3-3.9.18-3.el9.x86_64 12/73 2024-03-16T17:59:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : python-unversioned-command-3.9.18-3.el9.noarch 13/73 2024-03-16T17:59:56.948 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : git-core-doc-2.43.0-1.el9.noarch 14/73 2024-03-16T17:59:56.993 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : perl-Git-2.43.0-1.el9.noarch 15/73 2024-03-16T17:59:57.050 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : git-2.43.0-1.el9.x86_64 16/73 2024-03-16T17:59:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-03-16T17:59:57.522 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: device-mapper-event-9:1.02.197-2.el9.x86_64 17/73 2024-03-16T17:59:58.250 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : lvm2-libs-9:2.03.23-2.el9.x86_64 18/73 2024-03-16T17:59:58.334 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libstdc++-devel-11.4.1-3.el9.x86_64 19/73 2024-03-16T17:59:58.587 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libss-1.46.5-5.el9.x86_64 20/73 2024-03-16T17:59:58.795 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : cpp-11.4.1-3.el9.x86_64 21/73 2024-03-16T17:59:59.963 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : libgomp-11.4.1-3.el9.x86_64 22/73 2024-03-16T18:00:00.163 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : gcc-11.4.1-3.el9.x86_64 23/73 2024-03-16T18:00:00.242 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: gcc-11.4.1-3.el9.x86_64 23/73 2024-03-16T18:00:00.286 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 24/73 2024-03-16T18:00:00.317 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 25/73 2024-03-16T18:00:00.351 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 26/73 2024-03-16T18:00:00.385 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 27/73 2024-03-16T18:00:00.419 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 28/73 2024-03-16T18:00:00.453 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 29/73 2024-03-16T18:00:00.486 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 30/73 2024-03-16T18:00:00.733 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 31/73 2024-03-16T18:00:00.774 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fio-3.35-1.el9.x86_64 32/73 2024-03-16T18:00:00.812 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rmt-2:1.6-6.el9.x86_64 33/73 2024-03-16T18:00:00.868 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 34/73 2024-03-16T18:00:05.957 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:00:05.966 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:00:12.221 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 35/73 2024-03-16T18:00:12.477 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : kernel-devel-5.14.0-427.el9.x86_64 36/73 2024-03-16T18:00:36.100 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:00:36.109 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:01:02.029 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: kernel-devel-5.14.0-427.el9.x86_64 36/73 2024-03-16T18:01:02.107 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : gcc-c++-11.4.1-3.el9.x86_64 37/73 2024-03-16T18:01:02.146 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-03-16T18:01:02.317 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-3.el9.x86_64 38/73 2024-03-16T18:01:02.574 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : e2fsprogs-1.46.5-5.el9.x86_64 39/73 2024-03-16T18:01:02.574 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-03-16T18:01:02.574 INFO:teuthology.orchestra.run.smithi082.stdout:warning: /etc/lvm/lvm.conf created as /etc/lvm/lvm.conf.rpmnew 2024-03-16T18:01:02.574 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:01:03.078 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: lvm2-9:2.03.23-2.el9.x86_64 40/73 2024-03-16T18:01:03.131 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : python3-devel-3.9.18-3.el9.x86_64 41/73 2024-03-16T18:01:03.195 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 42/73 2024-03-16T18:01:03.249 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : indent-2.2.13-5.el9.x86_64 43/73 2024-03-16T18:01:03.321 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : udftools-2.2-5.el9.x86_64 44/73 2024-03-16T18:01:03.350 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 45/73 2024-03-16T18:01:03.389 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : git-2.39.3-1.el9.x86_64 46/73 2024-03-16T18:01:03.433 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : python3-devel-3.9.17-2.el9.x86_64 47/73 2024-03-16T18:01:03.471 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : e2fsprogs-1.46.5-3.el9.x86_64 48/73 2024-03-16T18:01:03.503 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : gcc-c++-11.4.1-2.1.el9.x86_64 49/73 2024-03-16T18:01:03.536 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : python3-3.9.17-2.el9.x86_64 50/73 2024-03-16T18:01:03.553 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : python-unversioned-command-3.9.17-2.el9.noarch 51/73 2024-03-16T18:01:03.554 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 52/73 2024-03-16T18:01:03.664 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 52/73 2024-03-16T18:01:03.723 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : gcc-11.4.1-2.1.el9.x86_64 53/73 2024-03-16T18:01:03.758 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-03-16T18:01:03.758 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-03-16T18:01:03.886 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: lvm2-9:2.03.21-3.el9.x86_64 54/73 2024-03-16T18:01:03.978 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libstdc++-devel-11.4.1-2.1.el9.x86_64 55/73 2024-03-16T18:01:04.021 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : git-core-doc-2.39.3-1.el9.noarch 56/73 2024-03-16T18:01:04.048 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : perl-Git-2.39.3-1.el9.noarch 57/73 2024-03-16T18:01:04.081 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : lvm2-libs-9:2.03.21-3.el9.x86_64 58/73 2024-03-16T18:01:04.108 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: device-mapper-event-9:1.02.195-3.el9.x86_64 59/73 2024-03-16T18:01:04.152 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : device-mapper-event-9:1.02.195-3.el9.x86_64 59/73 2024-03-16T18:01:04.191 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libstdc++-11.4.1-2.1.el9.x86_64 60/73 2024-03-16T18:01:04.224 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 61/73 2024-03-16T18:01:04.258 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : e2fsprogs-libs-1.46.5-3.el9.x86_64 62/73 2024-03-16T18:01:04.291 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libss-1.46.5-3.el9.x86_64 63/73 2024-03-16T18:01:04.334 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : device-mapper-libs-9:1.02.195-3.el9.x86_64 64/73 2024-03-16T18:01:04.461 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : device-mapper-9:1.02.195-3.el9.x86_64 65/73 2024-03-16T18:01:04.520 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : python3-libs-3.9.17-2.el9.x86_64 66/73 2024-03-16T18:01:04.552 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : sqlite-libs-3.34.1-6.el9.x86_64 67/73 2024-03-16T18:01:04.578 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libcom_err-1.46.5-3.el9.x86_64 68/73 2024-03-16T18:01:04.579 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-03-16T18:01:04.736 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libgcc-11.4.1-2.1.el9.x86_64 69/73 2024-03-16T18:01:04.783 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : git-core-2.39.3-1.el9.x86_64 70/73 2024-03-16T18:01:04.823 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : cpp-11.4.1-2.1.el9.x86_64 71/73 2024-03-16T18:01:04.858 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : libgomp-11.4.1-2.1.el9.x86_64 72/73 2024-03-16T18:01:04.987 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : make-1:4.3-7.el9.x86_64 73/73 2024-03-16T18:01:06.232 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:01:06.241 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: make-1:4.3-7.el9.x86_64 73/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fio-3.35-1.el9.x86_64 1/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 2/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 3/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 4/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 5/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 6/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 7/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 8/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 9/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : udftools-2.2-5.el9.x86_64 10/73 2024-03-16T18:01:06.913 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 11/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : kernel-devel-5.14.0-427.el9.x86_64 12/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 13/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python-unversioned-command-3.9.18-3.el9.noarch 18/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python-unversioned-command-3.9.17-2.el9.noarch 19/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 20/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 21/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-3.9.18-3.el9.x86_64 22/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-3.9.17-2.el9.x86_64 23/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-libs-3.9.18-3.el9.x86_64 24/73 2024-03-16T18:01:06.914 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-libs-3.9.17-2.el9.x86_64 25/73 2024-03-16T18:01:06.920 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : sqlite-libs-3.34.1-7.el9.x86_64 26/73 2024-03-16T18:01:06.920 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : sqlite-libs-3.34.1-6.el9.x86_64 27/73 2024-03-16T18:01:06.920 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : device-mapper-9:1.02.197-2.el9.x86_64 28/73 2024-03-16T18:01:06.920 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : device-mapper-9:1.02.195-3.el9.x86_64 29/73 2024-03-16T18:01:06.920 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : device-mapper-event-9:1.02.197-2.el9.x86_64 30/73 2024-03-16T18:01:06.920 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : device-mapper-event-9:1.02.195-3.el9.x86_64 31/73 2024-03-16T18:01:06.920 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : device-mapper-event-libs-9:1.02.197-2.el9.x86_64 32/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : device-mapper-event-libs-9:1.02.195-3.el9.x86_64 33/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : device-mapper-libs-9:1.02.197-2.el9.x86_64 34/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : device-mapper-libs-9:1.02.195-3.el9.x86_64 35/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : e2fsprogs-1.46.5-5.el9.x86_64 36/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : e2fsprogs-1.46.5-3.el9.x86_64 37/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : e2fsprogs-libs-1.46.5-5.el9.x86_64 38/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : e2fsprogs-libs-1.46.5-3.el9.x86_64 39/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcom_err-1.46.5-5.el9.x86_64 40/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcom_err-1.46.5-3.el9.x86_64 41/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgcc-11.4.1-3.el9.x86_64 42/73 2024-03-16T18:01:06.921 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgcc-11.4.1-2.1.el9.x86_64 43/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgomp-11.4.1-3.el9.x86_64 44/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgomp-11.4.1-2.1.el9.x86_64 45/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libss-1.46.5-5.el9.x86_64 46/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libss-1.46.5-3.el9.x86_64 47/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libstdc++-11.4.1-3.el9.x86_64 48/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libstdc++-11.4.1-2.1.el9.x86_64 49/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lvm2-9:2.03.23-2.el9.x86_64 50/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lvm2-9:2.03.21-3.el9.x86_64 51/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lvm2-libs-9:2.03.23-2.el9.x86_64 52/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lvm2-libs-9:2.03.21-3.el9.x86_64 53/73 2024-03-16T18:01:06.922 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : make-1:4.3-8.el9.x86_64 54/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : make-1:4.3-7.el9.x86_64 55/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cpp-11.4.1-3.el9.x86_64 56/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cpp-11.4.1-2.1.el9.x86_64 57/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gcc-11.4.1-3.el9.x86_64 58/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gcc-11.4.1-2.1.el9.x86_64 59/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gcc-c++-11.4.1-3.el9.x86_64 60/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gcc-c++-11.4.1-2.1.el9.x86_64 61/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gcc-plugin-annobin-11.4.1-3.el9.x86_64 62/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 63/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : git-2.43.0-1.el9.x86_64 64/73 2024-03-16T18:01:06.923 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : git-2.39.3-1.el9.x86_64 65/73 2024-03-16T18:01:06.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : git-core-2.43.0-1.el9.x86_64 66/73 2024-03-16T18:01:06.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : git-core-2.39.3-1.el9.x86_64 67/73 2024-03-16T18:01:06.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : git-core-doc-2.43.0-1.el9.noarch 68/73 2024-03-16T18:01:06.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : git-core-doc-2.39.3-1.el9.noarch 69/73 2024-03-16T18:01:06.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libstdc++-devel-11.4.1-3.el9.x86_64 70/73 2024-03-16T18:01:06.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libstdc++-devel-11.4.1-2.1.el9.x86_64 71/73 2024-03-16T18:01:06.924 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : perl-Git-2.43.0-1.el9.noarch 72/73 2024-03-16T18:01:07.910 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : perl-Git-2.39.3-1.el9.noarch 73/73 2024-03-16T18:01:07.910 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:01:07.910 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2024-03-16T18:01:07.910 INFO:teuthology.orchestra.run.smithi082.stdout: cpp-11.4.1-3.el9.x86_64 2024-03-16T18:01:07.910 INFO:teuthology.orchestra.run.smithi082.stdout: device-mapper-9:1.02.197-2.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: device-mapper-event-9:1.02.197-2.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: device-mapper-event-libs-9:1.02.197-2.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: device-mapper-libs-9:1.02.197-2.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: e2fsprogs-1.46.5-5.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: e2fsprogs-libs-1.46.5-5.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: gcc-11.4.1-3.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: gcc-c++-11.4.1-3.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: gcc-plugin-annobin-11.4.1-3.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: git-2.43.0-1.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: git-core-2.43.0-1.el9.x86_64 2024-03-16T18:01:07.911 INFO:teuthology.orchestra.run.smithi082.stdout: git-core-doc-2.43.0-1.el9.noarch 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-1.46.5-5.el9.x86_64 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: libgcc-11.4.1-3.el9.x86_64 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: libgomp-11.4.1-3.el9.x86_64 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: libss-1.46.5-5.el9.x86_64 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: libstdc++-11.4.1-3.el9.x86_64 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: libstdc++-devel-11.4.1-3.el9.x86_64 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: lvm2-9:2.03.23-2.el9.x86_64 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: lvm2-libs-9:2.03.23-2.el9.x86_64 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: make-1:4.3-8.el9.x86_64 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: perl-Git-2.43.0-1.el9.noarch 2024-03-16T18:01:07.912 INFO:teuthology.orchestra.run.smithi082.stdout: python-unversioned-command-3.9.18-3.el9.noarch 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: python3-3.9.18-3.el9.x86_64 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: python3-libs-3.9.18-3.el9.x86_64 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: sqlite-libs-3.34.1-7.el9.x86_64 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-427.el9.x86_64 2024-03-16T18:01:07.913 INFO:teuthology.orchestra.run.smithi082.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-03-16T18:01:07.914 INFO:teuthology.orchestra.run.smithi082.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-03-16T18:01:07.914 INFO:teuthology.orchestra.run.smithi082.stdout: udftools-2.2-5.el9.x86_64 2024-03-16T18:01:07.914 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:01:07.914 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2024-03-16T18:01:08.695 DEBUG:teuthology.orchestra.run.smithi082:> sudo useradd -m fsgqa 2024-03-16T18:01:08.771 INFO:teuthology.orchestra.run.smithi082.stderr:useradd: user 'fsgqa' already exists 2024-03-16T18:01:08.774 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-03-16T18:01:08.775 DEBUG:teuthology.orchestra.run.smithi082:> sudo groupadd fsgqa 2024-03-16T18:01:08.844 INFO:teuthology.orchestra.run.smithi082.stderr:groupadd: group 'fsgqa' already exists 2024-03-16T18:01:08.848 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-03-16T18:01:08.848 DEBUG:teuthology.orchestra.run.smithi082:> sudo useradd fsgqa2 2024-03-16T18:01:09.464 DEBUG:teuthology.orchestra.run.smithi082:> sudo useradd 123456-fsgqa 2024-03-16T18:01:10.086 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-03-16T18:01:10.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.259+0000 7f8200a00640 1 Processor -- start 2024-03-16T18:01:10.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.259+0000 7f8200a00640 1 -- start start 2024-03-16T18:01:10.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.260+0000 7f8200a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f81fc070820 0x7f81fc15f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.260+0000 7f8200a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f81fc0cd660 0x7f81fc0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.260+0000 7f8200a00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81fc0cf3a0 con 0x7f81fc143e00 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.260+0000 7f8200a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc15f5c0 con 0x7f81fc070820 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.260+0000 7f8200a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc0cf610 con 0x7f81fc0cd660 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.261+0000 7f81fa800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f81fc070820 0x7f81fc15f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.261+0000 7f81fa800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f81fc070820 0x7f81fc15f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43232/0 (socket says 172.21.15.82:43232) 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.261+0000 7f81fa800640 1 -- 172.21.15.82:0/4081770927 learned_addr learned my addr 172.21.15.82:0/4081770927 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.261+0000 7f81fb200640 1 --2- 172.21.15.82:0/4081770927 >> v2:172.21.15.90:3300/0 conn(0x7f81fc0cd660 0x7f81fc0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.261+0000 7f81fa800640 1 -- 172.21.15.82:0/4081770927 >> v2:172.21.15.90:3300/0 conn(0x7f81fc0cd660 msgr2=0x7f81fc0cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.261+0000 7f81fa800640 1 --2- 172.21.15.82:0/4081770927 >> v2:172.21.15.90:3300/0 conn(0x7f81fc0cd660 0x7f81fc0cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.261+0000 7f81fa800640 1 -- 172.21.15.82:0/4081770927 >> v1:172.21.15.90:6789/0 conn(0x7f81fc143e00 legacy=0x7f81fc070c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:01:10.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.261+0000 7f81fa800640 1 -- 172.21.15.82:0/4081770927 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc0cf8f0 con 0x7f81fc070820 2024-03-16T18:01:10.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.262+0000 7f81fa800640 1 --2- 172.21.15.82:0/4081770927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 0x7f81fc15f080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f81fc072410 tx=0x7f81ec02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b3fef5837fa84d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:01:10.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f81f9400640 1 -- 172.21.15.82:0/4081770927 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec02c9f0 con 0x7f81fc070820 2024-03-16T18:01:10.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f81f9400640 1 -- 172.21.15.82:0/4081770927 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec02cb50 con 0x7f81fc070820 2024-03-16T18:01:10.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f81f9400640 1 -- 172.21.15.82:0/4081770927 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec0343f0 con 0x7f81fc070820 2024-03-16T18:01:10.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f8200a00640 1 -- 172.21.15.82:0/4081770927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 msgr2=0x7f81fc15f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:10.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f8200a00640 1 --2- 172.21.15.82:0/4081770927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 0x7f81fc15f080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f81fc072410 tx=0x7f81ec02be50 comp rx=0 tx=0).stop 2024-03-16T18:01:10.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f8200a00640 1 -- 172.21.15.82:0/4081770927 shutdown_connections 2024-03-16T18:01:10.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f8200a00640 1 --2- 172.21.15.82:0/4081770927 >> v2:172.21.15.90:3300/0 conn(0x7f81fc0cd660 0x7f81fc0cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f8200a00640 1 --2- 172.21.15.82:0/4081770927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 0x7f81fc15f080 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f8200a00640 1 -- 172.21.15.82:0/4081770927 >> 172.21.15.82:0/4081770927 conn(0x7f81fc06ae70 msgr2=0x7f81fc06b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:01:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f8200a00640 1 -- 172.21.15.82:0/4081770927 shutdown_connections 2024-03-16T18:01:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.263+0000 7f8200a00640 1 -- 172.21.15.82:0/4081770927 wait complete. 2024-03-16T18:01:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.265+0000 7f8200a00640 1 Processor -- start 2024-03-16T18:01:10.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.265+0000 7f8200a00640 1 -- start start 2024-03-16T18:01:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.266+0000 7f8200a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 0x7f81fc0aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.266+0000 7f8200a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc0cd660 0x7f81fc0b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.266+0000 7f8200a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81fc143e00 0x7f81fc1f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.266+0000 7f8200a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc13e000 con 0x7f81fc070820 2024-03-16T18:01:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.266+0000 7f8200a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc1f83e0 con 0x7f81fc0cd660 2024-03-16T18:01:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.266+0000 7f8200a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc13da40 con 0x7f81fc143e00 2024-03-16T18:01:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.266+0000 7f81fa800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 0x7f81fc0aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.266+0000 7f81fa800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 0x7f81fc0aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43248/0 (socket says 172.21.15.82:43248) 2024-03-16T18:01:10.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.266+0000 7f81fa800640 1 -- 172.21.15.82:0/237288158 learned_addr learned my addr 172.21.15.82:0/237288158 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:01:10.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.267+0000 7f81f9e00640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc0cd660 0x7f81fc0b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.267+0000 7f81fa800640 1 -- 172.21.15.82:0/237288158 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81fc143e00 msgr2=0x7f81fc1f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.267+0000 7f81fa800640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81fc143e00 0x7f81fc1f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.267+0000 7f81fa800640 1 -- 172.21.15.82:0/237288158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc0cd660 msgr2=0x7f81fc0b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.267+0000 7f81fa800640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc0cd660 0x7f81fc0b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.267+0000 7f81fa800640 1 -- 172.21.15.82:0/237288158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec009b30 con 0x7f81fc070820 2024-03-16T18:01:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.267+0000 7f81fa800640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 0x7f81fc0aee90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f81ec000f80 tx=0x7f81ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:01:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.267+0000 7f81f2a00640 1 -- 172.21.15.82:0/237288158 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec03e450 con 0x7f81fc070820 2024-03-16T18:01:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.268+0000 7f81f2a00640 1 -- 172.21.15.82:0/237288158 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec03e5f0 con 0x7f81fc070820 2024-03-16T18:01:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.268+0000 7f81f2a00640 1 -- 172.21.15.82:0/237288158 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec03e9a0 con 0x7f81fc070820 2024-03-16T18:01:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.268+0000 7f8200a00640 1 -- 172.21.15.82:0/237288158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc13dd20 con 0x7f81fc070820 2024-03-16T18:01:10.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.268+0000 7f8200a00640 1 -- 172.21.15.82:0/237288158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc13fa10 con 0x7f81fc070820 2024-03-16T18:01:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.269+0000 7f8200a00640 1 -- 172.21.15.82:0/237288158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81fc0b4d80 con 0x7f81fc070820 2024-03-16T18:01:10.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.270+0000 7f81f2a00640 1 -- 172.21.15.82:0/237288158 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f81ec03eb00 con 0x7f81fc070820 2024-03-16T18:01:10.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.271+0000 7f81f2a00640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81d806b3b0 0x7f81d806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.272+0000 7f81f2a00640 1 -- 172.21.15.82:0/237288158 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81ec0acbe0 con 0x7f81fc070820 2024-03-16T18:01:10.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.272+0000 7f81f9e00640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81d806b3b0 0x7f81d806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.272+0000 7f81f9e00640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81d806b3b0 0x7f81d806d870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f81e4006fd0 tx=0x7f81e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:01:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.276+0000 7f81f2a00640 1 -- 172.21.15.82:0/237288158 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f81ec0778c0 con 0x7f81fc070820 2024-03-16T18:01:10.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.544+0000 7f8200a00640 1 -- 172.21.15.82:0/237288158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f81fc04f250 con 0x7f81fc070820 2024-03-16T18:01:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.544+0000 7f81f2a00640 1 -- 172.21.15.82:0/237288158 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1277 (secure 0 0 0) 0x7f81ec03edb0 con 0x7f81fc070820 2024-03-16T18:01:10.548 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:01:10.548 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":1,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","modified":"2024-03-16T17:56:37.619841Z","created":"2024-03-16T17:56:37.619841Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3301","nonce":0},{"type":"v1","addr":"172.21.15.90:6790","nonce":0}]},"addr":"172.21.15.90:6790/0","public_addr":"172.21.15.90:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-16T18:01:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:dumped monmap epoch 1 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 -- 172.21.15.82:0/237288158 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81d806b3b0 msgr2=0x7f81d806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81d806b3b0 0x7f81d806d870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f81e4006fd0 tx=0x7f81e4008040 comp rx=0 tx=0).stop 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 -- 172.21.15.82:0/237288158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 msgr2=0x7f81fc0aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 0x7f81fc0aee90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f81ec000f80 tx=0x7f81ec004a40 comp rx=0 tx=0).stop 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 -- 172.21.15.82:0/237288158 shutdown_connections 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f81d806b3b0 0x7f81d806d870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f81fc143e00 0x7f81fc1f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81fc0cd660 0x7f81fc0b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 --2- 172.21.15.82:0/237288158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81fc070820 0x7f81fc0aee90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 -- 172.21.15.82:0/237288158 >> 172.21.15.82:0/237288158 conn(0x7f81fc06ae70 msgr2=0x7f81fc06b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:01:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 -- 172.21.15.82:0/237288158 shutdown_connections 2024-03-16T18:01:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.547+0000 7f8200a00640 1 -- 172.21.15.82:0/237288158 wait complete. 2024-03-16T18:01:10.570 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.admin 2024-03-16T18:01:10.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.747+0000 7fcbdd400640 1 Processor -- start 2024-03-16T18:01:10.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.748+0000 7fcbdd400640 1 -- start start 2024-03-16T18:01:10.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.749+0000 7fcbdd400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbd80b71f0 0x7fcbd80d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.749+0000 7fcbdd400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd80b45d0 con 0x7fcbd806d0a0 2024-03-16T18:01:10.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.749+0000 7fcbdd400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd80b4980 con 0x7fcbd806c190 2024-03-16T18:01:10.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.749+0000 7fcbdd400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80b5290 con 0x7fcbd80b71f0 2024-03-16T18:01:10.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.749+0000 7fcbd7200640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcbd806d0a0 0x7fcbd80b56b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59736/0 (socket says 172.21.15.82:59736) 2024-03-16T18:01:10.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.749+0000 7fcbd7200640 1 -- 172.21.15.82:0/3154672759 learned_addr learned my addr 172.21.15.82:0/3154672759 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:01:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd6800640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbd80b71f0 0x7fcbd80d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557698545 0 0) 0x7fcbd0001490 con 0x7fcbd806d0a0 2024-03-16T18:01:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbbc003520 con 0x7fcbd806d0a0 2024-03-16T18:01:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 117157437 0 0) 0x7fcbbc003520 con 0x7fcbd806d0a0 2024-03-16T18:01:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbd80b71f0 msgr2=0x7fcbd80d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:01:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbd80b71f0 0x7fcbd80d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 >> v1:172.21.15.90:6789/0 conn(0x7fcbd806c190 legacy=0x7fcbd806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd80d0e80 con 0x7fcbd806d0a0 2024-03-16T18:01:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fcbd0001b80 con 0x7fcbd806d0a0 2024-03-16T18:01:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 >> v1:172.21.15.82:6789/0 conn(0x7fcbd806d0a0 legacy=0x7fcbd80b56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbd80b71f0 0x7fcbbc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbbc003f20 0x7fcbbc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbbc004d00 0x7fcbbc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80b5290 con 0x7fcbbc003f20 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc00bb10 con 0x7fcbbc004d00 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.750+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc00bcd0 con 0x7fcbd80b71f0 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.751+0000 7fcbd5e00640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbbc003f20 0x7fcbbc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.751+0000 7fcbd7200640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbbc004d00 0x7fcbbc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.751+0000 7fcbd5e00640 1 -- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbd80b71f0 msgr2=0x7fcbbc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.751+0000 7fcbd5e00640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbd80b71f0 0x7fcbbc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.751+0000 7fcbd5e00640 1 -- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbbc004d00 msgr2=0x7fcbbc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.751+0000 7fcbd5e00640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbbc004d00 0x7fcbbc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.751+0000 7fcbd5e00640 1 -- 172.21.15.82:0/3154672759 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc003520 con 0x7fcbbc003f20 2024-03-16T18:01:10.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.751+0000 7fcbd7200640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbbc004d00 0x7fcbbc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:01:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.752+0000 7fcbd5e00640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbbc003f20 0x7fcbbc0047c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4001750 tx=0x7fcbbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325d9b0df567b302 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:01:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.752+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc40052d0 con 0x7fcbbc003f20 2024-03-16T18:01:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.752+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc402e050 con 0x7fcbbc003f20 2024-03-16T18:01:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.752+0000 7fcbd5400640 1 -- 172.21.15.82:0/3154672759 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc40035f0 con 0x7fcbbc003f20 2024-03-16T18:01:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.753+0000 7fcbdd400640 1 -- 172.21.15.82:0/3154672759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbbc003f20 msgr2=0x7fcbbc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.753+0000 7fcbdd400640 1 --2- 172.21.15.82:0/3154672759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbbc003f20 0x7fcbbc0047c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4001750 tx=0x7fcbbc0057f0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.753+0000 7fcbd7200640 1 -- 172.21.15.82:0/3154672759 reap_dead start 2024-03-16T18:01:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.753+0000 7fcbdd400640 1 -- 172.21.15.82:0/3154672759 shutdown_connections 2024-03-16T18:01:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.753+0000 7fcbdd400640 1 -- 172.21.15.82:0/3154672759 >> 172.21.15.82:0/3154672759 conn(0x7fcbd80b3680 msgr2=0x7fcbd80b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:01:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.753+0000 7fcbdd400640 1 -- 172.21.15.82:0/3154672759 shutdown_connections 2024-03-16T18:01:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.753+0000 7fcbdd400640 1 -- 172.21.15.82:0/3154672759 wait complete. 2024-03-16T18:01:10.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.756+0000 7fcbdd400640 1 Processor -- start 2024-03-16T18:01:10.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.756+0000 7fcbdd400640 1 -- start start 2024-03-16T18:01:10.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.757+0000 7fcbdd400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbbc003f20 0x7fcbd80b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.757+0000 7fcbdd400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbd809e150 0x7fcbd809f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.757+0000 7fcbdd400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd80b07e0 0x7fcbd80c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.757+0000 7fcbdd400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80b3140 con 0x7fcbd809e150 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.757+0000 7fcbdd400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80b2e60 con 0x7fcbd80b07e0 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.757+0000 7fcbdd400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd809d670 con 0x7fcbbc003f20 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd5e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbd809e150 0x7fcbd809f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd5e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbd809e150 0x7fcbd809f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43268/0 (socket says 172.21.15.82:43268) 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd5e00640 1 -- 172.21.15.82:0/2636208185 learned_addr learned my addr 172.21.15.82:0/2636208185 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd6800640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbbc003f20 0x7fcbd80b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd7200640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd80b07e0 0x7fcbd80c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd5e00640 1 -- 172.21.15.82:0/2636208185 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbbc003f20 msgr2=0x7fcbd80b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd5e00640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbbc003f20 0x7fcbd80b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd5e00640 1 -- 172.21.15.82:0/2636208185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd80b07e0 msgr2=0x7fcbd80c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:10.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd5e00640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd80b07e0 0x7fcbd80c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:10.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd5e00640 1 -- 172.21.15.82:0/2636208185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd809d950 con 0x7fcbd809e150 2024-03-16T18:01:10.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.758+0000 7fcbd5e00640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbd809e150 0x7fcbd809f180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4008fd0 tx=0x7fcbc4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:01:10.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.759+0000 7fcbcea00640 1 -- 172.21.15.82:0/2636208185 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4007a30 con 0x7fcbd809e150 2024-03-16T18:01:10.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.759+0000 7fcbcea00640 1 -- 172.21.15.82:0/2636208185 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc4007640 con 0x7fcbd809e150 2024-03-16T18:01:10.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.759+0000 7fcbcea00640 1 -- 172.21.15.82:0/2636208185 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4006c20 con 0x7fcbd809e150 2024-03-16T18:01:10.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.759+0000 7fcbdd400640 1 -- 172.21.15.82:0/2636208185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd809f6c0 con 0x7fcbd809e150 2024-03-16T18:01:10.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.759+0000 7fcbdd400640 1 -- 172.21.15.82:0/2636208185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd80b0e00 con 0x7fcbd809e150 2024-03-16T18:01:10.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.760+0000 7fcbdd400640 1 -- 172.21.15.82:0/2636208185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd809d4d0 con 0x7fcbd809e150 2024-03-16T18:01:10.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.761+0000 7fcbcea00640 1 -- 172.21.15.82:0/2636208185 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fcbc40035f0 con 0x7fcbd809e150 2024-03-16T18:01:10.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.762+0000 7fcbcea00640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcbb806b3b0 0x7fcbb806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:01:10.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.762+0000 7fcbcea00640 1 -- 172.21.15.82:0/2636208185 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbc40ab950 con 0x7fcbd809e150 2024-03-16T18:01:10.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.762+0000 7fcbd6800640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcbb806b3b0 0x7fcbb806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:01:10.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.763+0000 7fcbd6800640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcbb806b3b0 0x7fcbb806d870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009a50 tx=0x7fcbc8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:01:10.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:10.766+0000 7fcbcea00640 1 -- 172.21.15.82:0/2636208185 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcbd809d4d0 con 0x7fcbd809e150 2024-03-16T18:01:11.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.036+0000 7fcbdd400640 1 -- 172.21.15.82:0/2636208185 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.admin"} v 0) v1 -- 0x7fcbd809d4d0 con 0x7fcbd809e150 2024-03-16T18:01:11.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.036+0000 7fcbcea00640 1 -- 172.21.15.82:0/2636208185 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.admin"}]=0 v13) v1 ==== 92+0+63 (secure 0 0 0) 0x7fcbd809d4d0 con 0x7fcbd809e150 2024-03-16T18:01:11.040 INFO:teuthology.orchestra.run.smithi082.stdout:[client.admin] 2024-03-16T18:01:11.040 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQBW3fVlHzMbAhAAPrAs7qHZeUgs4cGOFpD7ow== 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 -- 172.21.15.82:0/2636208185 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcbb806b3b0 msgr2=0x7fcbb806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcbb806b3b0 0x7fcbb806d870 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009a50 tx=0x7fcbc8008040 comp rx=0 tx=0).stop 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 -- 172.21.15.82:0/2636208185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbd809e150 msgr2=0x7fcbd809f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbd809e150 0x7fcbd809f180 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4008fd0 tx=0x7fcbc4001120 comp rx=0 tx=0).stop 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 -- 172.21.15.82:0/2636208185 shutdown_connections 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fcbb806b3b0 0x7fcbb806d870 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcbd80b07e0 0x7fcbd80c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbd809e150 0x7fcbd809f180 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 --2- 172.21.15.82:0/2636208185 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcbbc003f20 0x7fcbd80b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 -- 172.21.15.82:0/2636208185 >> 172.21.15.82:0/2636208185 conn(0x7fcbd80b3680 msgr2=0x7fcbd80c8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:01:11.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 -- 172.21.15.82:0/2636208185 shutdown_connections 2024-03-16T18:01:11.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:01:11.039+0000 7fcbdd400640 1 -- 172.21.15.82:0/2636208185 wait complete. 2024-03-16T18:01:11.063 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T18:01:11.063 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/tmp/tmp.YEFrEDdeoexfstests-dev/local.config 2024-03-16T18:01:11.160 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph export TEST_DEV=172.21.15.82:6789:/test export TEST_DIR=/tmp/tmp.8INGWKbrFetest export SCRATCH_DEV=172.21.15.82:6789:/scratch export SCRATCH_MNT=/tmp/tmp.npBvz5wtGascratch export CEPHFS_MOUNT_OPTIONS="-o name=admin,secret=AQBW3fVlHzMbAhAAPrAs7qHZeUgs4cGOFpD7ow==" 2024-03-16T18:01:11.160 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T18:01:11.160 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/tmp/tmp.YEFrEDdeoexfstests-dev/ceph.exclude 2024-03-16T18:01:11.281 DEBUG:teuthology.orchestra.run.smithi082:> (cd /tmp/tmp.YEFrEDdeoexfstests-dev && exec sudo make) 2024-03-16T18:01:11.406 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize -cfi 2024-03-16T18:01:12.004 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: putting auxiliary files in '.'. 2024-03-16T18:01:12.004 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: copying file './config.guess' 2024-03-16T18:01:12.014 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: copying file './config.sub' 2024-03-16T18:01:12.025 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: copying file './install-sh' 2024-03-16T18:01:12.036 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: copying file './ltmain.sh' 2024-03-16T18:01:12.116 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-03-16T18:01:12.116 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-03-16T18:01:12.138 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-03-16T18:01:12.155 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-03-16T18:01:12.173 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-03-16T18:01:12.190 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-03-16T18:01:12.213 INFO:teuthology.orchestra.run.smithi082.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-03-16T18:01:12.218 INFO:teuthology.orchestra.run.smithi082.stdout:cp include/install-sh . 2024-03-16T18:01:12.221 INFO:teuthology.orchestra.run.smithi082.stdout:aclocal -I m4 2024-03-16T18:01:13.815 INFO:teuthology.orchestra.run.smithi082.stdout:autoheader 2024-03-16T18:01:14.232 INFO:teuthology.orchestra.run.smithi082.stdout:autoconf 2024-03-16T18:01:14.500 INFO:teuthology.orchestra.run.smithi082.stdout:./configure \ 2024-03-16T18:01:14.500 INFO:teuthology.orchestra.run.smithi082.stdout: --libexecdir=/usr/lib \ 2024-03-16T18:01:14.500 INFO:teuthology.orchestra.run.smithi082.stdout: --exec_prefix=/var/lib 2024-03-16T18:01:14.976 INFO:teuthology.orchestra.run.smithi082.stdout:checking build system type... x86_64-pc-linux-gnu 2024-03-16T18:01:14.976 INFO:teuthology.orchestra.run.smithi082.stdout:checking host system type... x86_64-pc-linux-gnu 2024-03-16T18:01:14.981 INFO:teuthology.orchestra.run.smithi082.stdout:checking how to print strings... printf 2024-03-16T18:01:14.981 INFO:teuthology.orchestra.run.smithi082.stdout:checking for gcc... gcc 2024-03-16T18:01:15.116 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether the C compiler works... yes 2024-03-16T18:01:15.117 INFO:teuthology.orchestra.run.smithi082.stdout:checking for C compiler default output file name... a.out 2024-03-16T18:01:15.185 INFO:teuthology.orchestra.run.smithi082.stdout:checking for suffix of executables... 2024-03-16T18:01:15.291 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether we are cross compiling... no 2024-03-16T18:01:15.341 INFO:teuthology.orchestra.run.smithi082.stdout:checking for suffix of object files... o 2024-03-16T18:01:15.388 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether we are using the GNU C compiler... yes 2024-03-16T18:01:15.435 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether gcc accepts -g... yes 2024-03-16T18:01:15.499 INFO:teuthology.orchestra.run.smithi082.stdout:checking for gcc option to accept ISO C89... none needed 2024-03-16T18:01:15.509 INFO:teuthology.orchestra.run.smithi082.stdout:checking for a sed that does not truncate output... /bin/sed 2024-03-16T18:01:15.518 INFO:teuthology.orchestra.run.smithi082.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-03-16T18:01:15.523 INFO:teuthology.orchestra.run.smithi082.stdout:checking for egrep... /bin/grep -E 2024-03-16T18:01:15.529 INFO:teuthology.orchestra.run.smithi082.stdout:checking for fgrep... /bin/grep -F 2024-03-16T18:01:15.542 INFO:teuthology.orchestra.run.smithi082.stdout:checking for ld used by gcc... /bin/ld 2024-03-16T18:01:15.548 INFO:teuthology.orchestra.run.smithi082.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-03-16T18:01:15.559 INFO:teuthology.orchestra.run.smithi082.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-03-16T18:01:16.600 INFO:teuthology.orchestra.run.smithi082.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-03-16T18:01:16.601 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether ln -s works... yes 2024-03-16T18:01:16.624 INFO:teuthology.orchestra.run.smithi082.stdout:checking the maximum length of command line arguments... 1572864 2024-03-16T18:01:16.634 INFO:teuthology.orchestra.run.smithi082.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-03-16T18:01:16.635 INFO:teuthology.orchestra.run.smithi082.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-03-16T18:01:16.635 INFO:teuthology.orchestra.run.smithi082.stdout:checking for /bin/ld option to reload object files... -r 2024-03-16T18:01:16.635 INFO:teuthology.orchestra.run.smithi082.stdout:checking for objdump... objdump 2024-03-16T18:01:16.635 INFO:teuthology.orchestra.run.smithi082.stdout:checking how to recognize dependent libraries... pass_all 2024-03-16T18:01:16.635 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dlltool... no 2024-03-16T18:01:16.635 INFO:teuthology.orchestra.run.smithi082.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-03-16T18:01:16.635 INFO:teuthology.orchestra.run.smithi082.stdout:checking for ar... ar 2024-03-16T18:01:16.742 INFO:teuthology.orchestra.run.smithi082.stdout:checking for archiver @FILE support... @ 2024-03-16T18:01:16.743 INFO:teuthology.orchestra.run.smithi082.stdout:checking for strip... strip 2024-03-16T18:01:16.743 INFO:teuthology.orchestra.run.smithi082.stdout:checking for ranlib... ranlib 2024-03-16T18:01:16.743 INFO:teuthology.orchestra.run.smithi082.stdout:checking for gawk... gawk 2024-03-16T18:01:16.914 INFO:teuthology.orchestra.run.smithi082.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-03-16T18:01:16.919 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sysroot... no 2024-03-16T18:01:16.928 INFO:teuthology.orchestra.run.smithi082.stdout:checking for a working dd... /bin/dd 2024-03-16T18:01:16.938 INFO:teuthology.orchestra.run.smithi082.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-03-16T18:01:16.975 INFO:teuthology.orchestra.run.smithi082.stdout:checking for mt... no 2024-03-16T18:01:16.982 INFO:teuthology.orchestra.run.smithi082.stdout:checking if : is a manifest tool... no 2024-03-16T18:01:17.073 INFO:teuthology.orchestra.run.smithi082.stdout:checking how to run the C preprocessor... gcc -E 2024-03-16T18:01:17.420 INFO:teuthology.orchestra.run.smithi082.stdout:checking for ANSI C header files... yes 2024-03-16T18:01:17.474 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/types.h... yes 2024-03-16T18:01:17.545 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/stat.h... yes 2024-03-16T18:01:17.615 INFO:teuthology.orchestra.run.smithi082.stdout:checking for stdlib.h... yes 2024-03-16T18:01:17.683 INFO:teuthology.orchestra.run.smithi082.stdout:checking for string.h... yes 2024-03-16T18:01:17.757 INFO:teuthology.orchestra.run.smithi082.stdout:checking for memory.h... yes 2024-03-16T18:01:17.825 INFO:teuthology.orchestra.run.smithi082.stdout:checking for strings.h... yes 2024-03-16T18:01:17.908 INFO:teuthology.orchestra.run.smithi082.stdout:checking for inttypes.h... yes 2024-03-16T18:01:17.981 INFO:teuthology.orchestra.run.smithi082.stdout:checking for stdint.h... yes 2024-03-16T18:01:18.067 INFO:teuthology.orchestra.run.smithi082.stdout:checking for unistd.h... yes 2024-03-16T18:01:18.142 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dlfcn.h... yes 2024-03-16T18:01:18.155 INFO:teuthology.orchestra.run.smithi082.stdout:checking for objdir... .libs 2024-03-16T18:01:18.336 INFO:teuthology.orchestra.run.smithi082.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-03-16T18:01:18.336 INFO:teuthology.orchestra.run.smithi082.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-03-16T18:01:18.389 INFO:teuthology.orchestra.run.smithi082.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-03-16T18:01:18.442 INFO:teuthology.orchestra.run.smithi082.stdout:checking if gcc static flag -static works... no 2024-03-16T18:01:18.508 INFO:teuthology.orchestra.run.smithi082.stdout:checking if gcc supports -c -o file.o... yes 2024-03-16T18:01:18.508 INFO:teuthology.orchestra.run.smithi082.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-03-16T18:01:18.527 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-03-16T18:01:18.589 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether -lc should be explicitly linked in... no 2024-03-16T18:01:18.719 INFO:teuthology.orchestra.run.smithi082.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-03-16T18:01:18.719 INFO:teuthology.orchestra.run.smithi082.stdout:checking how to hardcode library paths into programs... immediate 2024-03-16T18:01:18.722 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether stripping libraries is possible... yes 2024-03-16T18:01:18.722 INFO:teuthology.orchestra.run.smithi082.stdout:checking if libtool supports shared libraries... yes 2024-03-16T18:01:18.722 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether to build shared libraries... yes 2024-03-16T18:01:18.722 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether to build static libraries... yes 2024-03-16T18:01:18.741 INFO:teuthology.orchestra.run.smithi082.stdout:checking for gcc... (cached) gcc 2024-03-16T18:01:18.791 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-03-16T18:01:18.791 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether gcc accepts -g... (cached) yes 2024-03-16T18:01:18.791 INFO:teuthology.orchestra.run.smithi082.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-03-16T18:01:18.791 INFO:teuthology.orchestra.run.smithi082.stdout:checking for gmake... /usr/bin/gmake 2024-03-16T18:01:18.792 INFO:teuthology.orchestra.run.smithi082.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-03-16T18:01:18.793 INFO:teuthology.orchestra.run.smithi082.stdout:checking for tar... /bin/tar 2024-03-16T18:01:18.793 INFO:teuthology.orchestra.run.smithi082.stdout:checking for gzip... /bin/gzip 2024-03-16T18:01:18.818 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether gcc -MM is supported... yes 2024-03-16T18:01:18.819 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sort... /bin/sort 2024-03-16T18:01:18.819 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether ln -s works... yes 2024-03-16T18:01:18.819 INFO:teuthology.orchestra.run.smithi082.stdout:checking for rpm... /bin/rpm 2024-03-16T18:01:18.828 INFO:teuthology.orchestra.run.smithi082.stdout:checking for rpmbuild... no 2024-03-16T18:01:18.889 INFO:teuthology.orchestra.run.smithi082.stdout:checking assert.h usability... yes 2024-03-16T18:01:18.915 INFO:teuthology.orchestra.run.smithi082.stdout:checking assert.h presence... yes 2024-03-16T18:01:18.916 INFO:teuthology.orchestra.run.smithi082.stdout:checking for assert.h... yes 2024-03-16T18:01:18.984 INFO:teuthology.orchestra.run.smithi082.stdout:checking bstring.h usability... no 2024-03-16T18:01:19.025 INFO:teuthology.orchestra.run.smithi082.stdout:checking bstring.h presence... no 2024-03-16T18:01:19.025 INFO:teuthology.orchestra.run.smithi082.stdout:checking for bstring.h... no 2024-03-16T18:01:19.095 INFO:teuthology.orchestra.run.smithi082.stdout:checking libgen.h usability... yes 2024-03-16T18:01:19.120 INFO:teuthology.orchestra.run.smithi082.stdout:checking libgen.h presence... yes 2024-03-16T18:01:19.121 INFO:teuthology.orchestra.run.smithi082.stdout:checking for libgen.h... yes 2024-03-16T18:01:19.207 INFO:teuthology.orchestra.run.smithi082.stdout:checking dirent.h usability... yes 2024-03-16T18:01:19.235 INFO:teuthology.orchestra.run.smithi082.stdout:checking dirent.h presence... yes 2024-03-16T18:01:19.235 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dirent.h... yes 2024-03-16T18:01:19.336 INFO:teuthology.orchestra.run.smithi082.stdout:checking errno.h usability... yes 2024-03-16T18:01:19.362 INFO:teuthology.orchestra.run.smithi082.stdout:checking errno.h presence... yes 2024-03-16T18:01:19.362 INFO:teuthology.orchestra.run.smithi082.stdout:checking for errno.h... yes 2024-03-16T18:01:19.472 INFO:teuthology.orchestra.run.smithi082.stdout:checking malloc.h usability... yes 2024-03-16T18:01:19.500 INFO:teuthology.orchestra.run.smithi082.stdout:checking malloc.h presence... yes 2024-03-16T18:01:19.500 INFO:teuthology.orchestra.run.smithi082.stdout:checking for malloc.h... yes 2024-03-16T18:01:19.577 INFO:teuthology.orchestra.run.smithi082.stdout:checking uuid.h usability... no 2024-03-16T18:01:19.618 INFO:teuthology.orchestra.run.smithi082.stdout:checking uuid.h presence... no 2024-03-16T18:01:19.618 INFO:teuthology.orchestra.run.smithi082.stdout:checking for uuid.h... no 2024-03-16T18:01:19.700 INFO:teuthology.orchestra.run.smithi082.stdout:checking uuid/uuid.h usability... yes 2024-03-16T18:01:19.730 INFO:teuthology.orchestra.run.smithi082.stdout:checking uuid/uuid.h presence... yes 2024-03-16T18:01:19.731 INFO:teuthology.orchestra.run.smithi082.stdout:checking for uuid/uuid.h... yes 2024-03-16T18:01:19.803 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/uuid.h usability... no 2024-03-16T18:01:19.840 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/uuid.h presence... no 2024-03-16T18:01:19.840 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/uuid.h... no 2024-03-16T18:01:19.910 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/file.h usability... yes 2024-03-16T18:01:19.937 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/file.h presence... yes 2024-03-16T18:01:19.937 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/file.h... yes 2024-03-16T18:01:20.022 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/fcntl.h usability... yes 2024-03-16T18:01:20.050 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/fcntl.h presence... yes 2024-03-16T18:01:20.051 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/fcntl.h... yes 2024-03-16T18:01:20.126 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/syssgi.h usability... no 2024-03-16T18:01:20.166 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/syssgi.h presence... no 2024-03-16T18:01:20.166 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/syssgi.h... no 2024-03-16T18:01:20.263 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/param.h usability... yes 2024-03-16T18:01:20.300 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/param.h presence... yes 2024-03-16T18:01:20.300 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/param.h... yes 2024-03-16T18:01:20.318 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/stat.h... (cached) yes 2024-03-16T18:01:20.400 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/statvfs.h usability... yes 2024-03-16T18:01:20.426 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/statvfs.h presence... yes 2024-03-16T18:01:20.426 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/statvfs.h... yes 2024-03-16T18:01:20.501 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/time.h usability... yes 2024-03-16T18:01:20.528 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/time.h presence... yes 2024-03-16T18:01:20.528 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/time.h... yes 2024-03-16T18:01:20.616 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/ioctl.h usability... yes 2024-03-16T18:01:20.644 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/ioctl.h presence... yes 2024-03-16T18:01:20.645 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/ioctl.h... yes 2024-03-16T18:01:20.726 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/wait.h usability... yes 2024-03-16T18:01:20.755 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/wait.h presence... yes 2024-03-16T18:01:20.755 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/wait.h... yes 2024-03-16T18:01:20.774 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/types.h... (cached) yes 2024-03-16T18:01:20.791 INFO:teuthology.orchestra.run.smithi082.stdout:checking for strings.h... (cached) yes 2024-03-16T18:01:20.860 INFO:teuthology.orchestra.run.smithi082.stdout:checking err.h usability... yes 2024-03-16T18:01:20.885 INFO:teuthology.orchestra.run.smithi082.stdout:checking err.h presence... yes 2024-03-16T18:01:20.885 INFO:teuthology.orchestra.run.smithi082.stdout:checking for err.h... yes 2024-03-16T18:01:20.970 INFO:teuthology.orchestra.run.smithi082.stdout:checking linux/falloc.h usability... yes 2024-03-16T18:01:20.994 INFO:teuthology.orchestra.run.smithi082.stdout:checking linux/falloc.h presence... yes 2024-03-16T18:01:20.995 INFO:teuthology.orchestra.run.smithi082.stdout:checking for linux/falloc.h... yes 2024-03-16T18:01:21.064 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-03-16T18:01:21.100 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-03-16T18:01:21.100 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/fs/xfs_fsops.h... no 2024-03-16T18:01:21.169 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/fs/xfs_itable.h usability... no 2024-03-16T18:01:21.205 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/fs/xfs_itable.h presence... no 2024-03-16T18:01:21.205 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/fs/xfs_itable.h... no 2024-03-16T18:01:21.272 INFO:teuthology.orchestra.run.smithi082.stdout:checking xfs/platform_defs.h usability... no 2024-03-16T18:01:21.313 INFO:teuthology.orchestra.run.smithi082.stdout:checking xfs/platform_defs.h presence... no 2024-03-16T18:01:21.313 INFO:teuthology.orchestra.run.smithi082.stdout:checking for xfs/platform_defs.h... no 2024-03-16T18:01:21.382 INFO:teuthology.orchestra.run.smithi082.stdout:checking btrfs/ioctl.h usability... no 2024-03-16T18:01:21.424 INFO:teuthology.orchestra.run.smithi082.stdout:checking btrfs/ioctl.h presence... no 2024-03-16T18:01:21.424 INFO:teuthology.orchestra.run.smithi082.stdout:checking for btrfs/ioctl.h... no 2024-03-16T18:01:21.494 INFO:teuthology.orchestra.run.smithi082.stdout:checking cifs/ioctl.h usability... no 2024-03-16T18:01:21.535 INFO:teuthology.orchestra.run.smithi082.stdout:checking cifs/ioctl.h presence... no 2024-03-16T18:01:21.536 INFO:teuthology.orchestra.run.smithi082.stdout:checking for cifs/ioctl.h... no 2024-03-16T18:01:21.606 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/mman.h usability... yes 2024-03-16T18:01:21.632 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/mman.h presence... yes 2024-03-16T18:01:21.632 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/mman.h... yes 2024-03-16T18:01:21.684 INFO:teuthology.orchestra.run.smithi082.stdout:checking for xfs/xfs_log_format.h... no 2024-03-16T18:01:21.780 INFO:teuthology.orchestra.run.smithi082.stdout:checking for uuid_compare... no 2024-03-16T18:01:21.861 INFO:teuthology.orchestra.run.smithi082.stdout:checking for uuid_compare in -luuid... yes 2024-03-16T18:01:21.975 INFO:teuthology.orchestra.run.smithi082.stdout:checking for xfs/xfs.h... yes 2024-03-16T18:01:22.018 INFO:teuthology.orchestra.run.smithi082.stdout:checking for xfs/libxfs.h... no 2024-03-16T18:01:22.070 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether xlog_assign_lsn is declared... no 2024-03-16T18:01:22.140 INFO:teuthology.orchestra.run.smithi082.stdout:checking for xfs/xqm.h... yes 2024-03-16T18:01:22.246 INFO:teuthology.orchestra.run.smithi082.stdout:checking xfsctl from xfs/xfs.h... ok 2024-03-16T18:01:22.308 INFO:teuthology.orchestra.run.smithi082.stdout:checking xfs/handle.h usability... yes 2024-03-16T18:01:22.331 INFO:teuthology.orchestra.run.smithi082.stdout:checking xfs/handle.h presence... yes 2024-03-16T18:01:22.331 INFO:teuthology.orchestra.run.smithi082.stdout:checking for xfs/handle.h... yes 2024-03-16T18:01:22.415 INFO:teuthology.orchestra.run.smithi082.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-03-16T18:01:22.480 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/acl.h usability... yes 2024-03-16T18:01:22.510 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/acl.h presence... yes 2024-03-16T18:01:22.511 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/acl.h... yes 2024-03-16T18:01:22.569 INFO:teuthology.orchestra.run.smithi082.stdout:checking acl/libacl.h usability... yes 2024-03-16T18:01:22.600 INFO:teuthology.orchestra.run.smithi082.stdout:checking acl/libacl.h presence... yes 2024-03-16T18:01:22.600 INFO:teuthology.orchestra.run.smithi082.stdout:checking for acl/libacl.h... yes 2024-03-16T18:01:22.686 INFO:teuthology.orchestra.run.smithi082.stdout:checking for acl_init in -lacl... yes 2024-03-16T18:01:22.755 INFO:teuthology.orchestra.run.smithi082.stdout:checking gdbm-ndbm.h usability... no 2024-03-16T18:01:22.797 INFO:teuthology.orchestra.run.smithi082.stdout:checking gdbm-ndbm.h presence... no 2024-03-16T18:01:22.797 INFO:teuthology.orchestra.run.smithi082.stdout:checking for gdbm-ndbm.h... no 2024-03-16T18:01:22.859 INFO:teuthology.orchestra.run.smithi082.stdout:checking gdbm/ndbm.h usability... yes 2024-03-16T18:01:22.888 INFO:teuthology.orchestra.run.smithi082.stdout:checking gdbm/ndbm.h presence... yes 2024-03-16T18:01:22.888 INFO:teuthology.orchestra.run.smithi082.stdout:checking for gdbm/ndbm.h... yes 2024-03-16T18:01:22.975 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dbm_open in -lgdbm... no 2024-03-16T18:01:23.066 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dbm_fetch in -lgdbm... no 2024-03-16T18:01:23.155 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dbm_store in -lgdbm... no 2024-03-16T18:01:23.246 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dbm_close in -lgdbm... no 2024-03-16T18:01:23.335 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-03-16T18:01:23.411 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-03-16T18:01:23.490 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-03-16T18:01:23.568 INFO:teuthology.orchestra.run.smithi082.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-03-16T18:01:23.629 INFO:teuthology.orchestra.run.smithi082.stdout:checking libaio.h usability... yes 2024-03-16T18:01:23.662 INFO:teuthology.orchestra.run.smithi082.stdout:checking libaio.h presence... yes 2024-03-16T18:01:23.662 INFO:teuthology.orchestra.run.smithi082.stdout:checking for libaio.h... yes 2024-03-16T18:01:23.726 INFO:teuthology.orchestra.run.smithi082.stdout:checking liburing.h usability... no 2024-03-16T18:01:23.767 INFO:teuthology.orchestra.run.smithi082.stdout:checking liburing.h presence... no 2024-03-16T18:01:23.767 INFO:teuthology.orchestra.run.smithi082.stdout:checking for liburing.h... no 2024-03-16T18:01:23.831 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/capability.h usability... yes 2024-03-16T18:01:23.861 INFO:teuthology.orchestra.run.smithi082.stdout:checking sys/capability.h presence... yes 2024-03-16T18:01:23.861 INFO:teuthology.orchestra.run.smithi082.stdout:checking for sys/capability.h... yes 2024-03-16T18:01:23.922 INFO:teuthology.orchestra.run.smithi082.stdout:checking linux/fiemap.h usability... yes 2024-03-16T18:01:23.946 INFO:teuthology.orchestra.run.smithi082.stdout:checking linux/fiemap.h presence... yes 2024-03-16T18:01:23.946 INFO:teuthology.orchestra.run.smithi082.stdout:checking for linux/fiemap.h... yes 2024-03-16T18:01:24.027 INFO:teuthology.orchestra.run.smithi082.stdout:checking for fallocate... yes 2024-03-16T18:01:24.111 INFO:teuthology.orchestra.run.smithi082.stdout:checking for open_by_handle_at... yes 2024-03-16T18:01:24.175 INFO:teuthology.orchestra.run.smithi082.stdout:checking linux/fs.h usability... yes 2024-03-16T18:01:24.204 INFO:teuthology.orchestra.run.smithi082.stdout:checking linux/fs.h presence... yes 2024-03-16T18:01:24.205 INFO:teuthology.orchestra.run.smithi082.stdout:checking for linux/fs.h... yes 2024-03-16T18:01:24.269 INFO:teuthology.orchestra.run.smithi082.stdout:checking btrfsutil.h usability... no 2024-03-16T18:01:24.311 INFO:teuthology.orchestra.run.smithi082.stdout:checking btrfsutil.h presence... no 2024-03-16T18:01:24.311 INFO:teuthology.orchestra.run.smithi082.stdout:checking for btrfsutil.h... no 2024-03-16T18:01:24.397 INFO:teuthology.orchestra.run.smithi082.stdout:checking for copy_file_range... yes 2024-03-16T18:01:24.480 INFO:teuthology.orchestra.run.smithi082.stdout:checking for SEEK_DATA... yes 2024-03-16T18:01:24.582 INFO:teuthology.orchestra.run.smithi082.stdout:checking for BMV_OF_SHARED... yes 2024-03-16T18:01:24.672 INFO:teuthology.orchestra.run.smithi082.stdout:checking for nftw... yes 2024-03-16T18:01:24.768 INFO:teuthology.orchestra.run.smithi082.stdout:checking for RLIMIT_NOFILE... yes 2024-03-16T18:01:24.849 INFO:teuthology.orchestra.run.smithi082.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-03-16T18:01:24.932 INFO:teuthology.orchestra.run.smithi082.stdout:checking for FICLONE... yes 2024-03-16T18:01:25.016 INFO:teuthology.orchestra.run.smithi082.stdout:checking for renameat2... yes 2024-03-16T18:01:25.100 INFO:teuthology.orchestra.run.smithi082.stdout:checking for reallocarray... yes 2024-03-16T18:01:25.195 INFO:teuthology.orchestra.run.smithi082.stdout:checking for struct mount_attr... yes 2024-03-16T18:01:25.297 INFO:teuthology.orchestra.run.smithi082.stdout:checking for struct btrfs_qgroup_limit... yes 2024-03-16T18:01:25.403 INFO:teuthology.orchestra.run.smithi082.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-03-16T18:01:25.505 INFO:teuthology.orchestra.run.smithi082.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-03-16T18:01:25.605 INFO:teuthology.orchestra.run.smithi082.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-03-16T18:01:25.704 INFO:teuthology.orchestra.run.smithi082.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-03-16T18:01:25.807 INFO:teuthology.orchestra.run.smithi082.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-03-16T18:01:25.911 INFO:teuthology.orchestra.run.smithi082.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-03-16T18:01:25.983 INFO:teuthology.orchestra.run.smithi082.stdout:checking linux/btrfs.h usability... yes 2024-03-16T18:01:26.010 INFO:teuthology.orchestra.run.smithi082.stdout:checking linux/btrfs.h presence... yes 2024-03-16T18:01:26.010 INFO:teuthology.orchestra.run.smithi082.stdout:checking for linux/btrfs.h... yes 2024-03-16T18:01:26.101 INFO:teuthology.orchestra.run.smithi082.stdout:checking linux/btrfs_tree.h usability... yes 2024-03-16T18:01:26.129 INFO:teuthology.orchestra.run.smithi082.stdout:checking linux/btrfs_tree.h presence... yes 2024-03-16T18:01:26.129 INFO:teuthology.orchestra.run.smithi082.stdout:checking for linux/btrfs_tree.h... yes 2024-03-16T18:01:26.193 INFO:teuthology.orchestra.run.smithi082.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-03-16T18:01:26.266 INFO:teuthology.orchestra.run.smithi082.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-03-16T18:01:26.307 INFO:teuthology.orchestra.run.smithi082.stdout:configure: creating ./config.status 2024-03-16T18:01:27.501 INFO:teuthology.orchestra.run.smithi082.stdout:config.status: creating include/builddefs 2024-03-16T18:01:27.541 INFO:teuthology.orchestra.run.smithi082.stdout:config.status: creating include/config.h 2024-03-16T18:01:27.572 INFO:teuthology.orchestra.run.smithi082.stdout:config.status: executing libtool commands 2024-03-16T18:01:27.673 INFO:teuthology.orchestra.run.smithi082.stdout:Building include 2024-03-16T18:01:27.683 INFO:teuthology.orchestra.run.smithi082.stdout:Building lib 2024-03-16T18:01:27.693 INFO:teuthology.orchestra.run.smithi082.stdout: [LTDEP] 2024-03-16T18:01:28.022 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] dataascii.lo 2024-03-16T18:01:28.285 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] databin.lo 2024-03-16T18:01:28.579 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] datapid.lo 2024-03-16T18:01:28.827 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] file_lock.lo 2024-03-16T18:01:29.118 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] forker.lo 2024-03-16T18:01:29.389 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] pattern.lo 2024-03-16T18:01:29.645 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] open_flags.lo 2024-03-16T18:01:29.934 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] random_range.lo 2024-03-16T18:01:30.308 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] string_to_tokens.lo 2024-03-16T18:01:30.535 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] str_to_bytes.lo 2024-03-16T18:01:30.812 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] tlibio.lo 2024-03-16T18:01:31.223 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] write_log.lo 2024-03-16T18:01:31.608 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] random.lo 2024-03-16T18:01:31.846 INFO:teuthology.orchestra.run.smithi082.stdout: [LD] 2024-03-16T18:01:32.052 INFO:teuthology.orchestra.run.smithi082.stdout:Building ltp 2024-03-16T18:01:32.061 INFO:teuthology.orchestra.run.smithi082.stdout: [DEP] 2024-03-16T18:01:32.332 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] doio 2024-03-16T18:01:33.216 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fsstress 2024-03-16T18:01:34.840 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fsx 2024-03-16T18:01:35.780 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] growfiles 2024-03-16T18:01:36.417 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] iogen 2024-03-16T18:01:36.675 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:01:36.683 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:01:36.984 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-stress 2024-03-16T18:01:37.621 INFO:teuthology.orchestra.run.smithi082.stdout:Building src 2024-03-16T18:01:37.634 INFO:teuthology.orchestra.run.smithi082.stdout: [DEP] 2024-03-16T18:01:41.502 INFO:teuthology.orchestra.run.smithi082.stdout: [DEP] 2024-03-16T18:01:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: [DEP] 2024-03-16T18:01:41.596 INFO:teuthology.orchestra.run.smithi082.stdout: [DEP] 2024-03-16T18:01:42.068 INFO:teuthology.orchestra.run.smithi082.stdout: [DEP] 2024-03-16T18:01:42.126 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] dirstress 2024-03-16T18:01:42.556 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fill 2024-03-16T18:01:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fill2 2024-03-16T18:01:43.338 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] getpagesize 2024-03-16T18:01:43.645 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] holes 2024-03-16T18:01:44.030 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] lstat64 2024-03-16T18:01:44.389 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] nametest 2024-03-16T18:01:44.833 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] permname 2024-03-16T18:01:45.219 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] randholes 2024-03-16T18:01:45.626 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] runas 2024-03-16T18:01:45.946 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] truncfile 2024-03-16T18:01:46.314 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] usemem 2024-03-16T18:01:46.626 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] mmapcat 2024-03-16T18:01:46.938 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] append_reader 2024-03-16T18:01:47.245 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] append_writer 2024-03-16T18:01:47.572 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] dirperf 2024-03-16T18:01:47.993 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] metaperf 2024-03-16T18:01:48.440 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] devzero 2024-03-16T18:01:48.802 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] feature 2024-03-16T18:01:49.173 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] alloc 2024-03-16T18:01:49.543 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fault 2024-03-16T18:01:49.887 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fstest 2024-03-16T18:01:50.285 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_access_root 2024-03-16T18:01:50.588 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] godown 2024-03-16T18:01:50.946 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] resvtest 2024-03-16T18:01:51.311 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] writemod 2024-03-16T18:01:51.651 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] writev_on_pagefault 2024-03-16T18:01:51.983 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] makeextents 2024-03-16T18:01:52.357 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] itrash 2024-03-16T18:01:52.693 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] rename 2024-03-16T18:01:52.985 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] multi_open_unlink 2024-03-16T18:01:53.311 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] unwritten_sync 2024-03-16T18:01:53.680 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] genhashnames 2024-03-16T18:01:54.050 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_holes 2024-03-16T18:01:54.360 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_mmap_writev 2024-03-16T18:01:54.666 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_truncate_cmtime 2024-03-16T18:01:54.995 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] dirhash_collide 2024-03-16T18:01:55.347 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_rename_overwrite 2024-03-16T18:01:55.654 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] holetest 2024-03-16T18:01:56.070 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_truncate_self 2024-03-16T18:01:56.380 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] af_unix 2024-03-16T18:01:56.687 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_mmap_stale_pmd 2024-03-16T18:01:56.999 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_mmap_cow_race 2024-03-16T18:01:57.342 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_mmap_fallocate 2024-03-16T18:01:57.666 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fsync-err 2024-03-16T18:01:58.053 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_mmap_write_ro 2024-03-16T18:01:58.387 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_ext4_dax_journal_corruption 2024-03-16T18:01:58.717 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_ext4_dax_inline_corruption 2024-03-16T18:01:59.066 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_ofd_locks 2024-03-16T18:01:59.443 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_mmap_collision 2024-03-16T18:01:59.823 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] mmap-write-concurrent 2024-03-16T18:02:00.172 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_get_file_time 2024-03-16T18:02:00.480 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_create_short_dirs 2024-03-16T18:02:00.812 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_create_long_dirs 2024-03-16T18:02:01.158 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_enospc 2024-03-16T18:02:01.547 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_mmap_writev_overlap 2024-03-16T18:02:01.914 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] checkpoint_journal 2024-03-16T18:02:02.235 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] mmap-rw-fault 2024-03-16T18:02:02.579 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] allocstale 2024-03-16T18:02:02.926 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_mmap_cow_memory_failure 2024-03-16T18:02:03.284 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fake-dump-rootino 2024-03-16T18:02:03.605 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] dio-buf-fault 2024-03-16T18:02:03.935 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] rewinddir-test 2024-03-16T18:02:04.258 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] readdir-while-renames 2024-03-16T18:02:04.600 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] xfsctl 2024-03-16T18:02:04.973 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] bstat 2024-03-16T18:02:05.366 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_mtab 2024-03-16T18:02:05.723 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] getdevicesize 2024-03-16T18:02:06.012 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] preallo_rw_pattern_reader 2024-03-16T18:02:06.354 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] preallo_rw_pattern_writer 2024-03-16T18:02:06.668 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] ftrunc 2024-03-16T18:02:06.802 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:02:06.814 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:02:06.949 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] trunc 2024-03-16T18:02:07.262 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fs_perms 2024-03-16T18:02:07.616 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] testx 2024-03-16T18:02:07.925 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] looptest 2024-03-16T18:02:08.288 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] locktest 2024-03-16T18:02:08.796 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] unwritten_mmap 2024-03-16T18:02:09.145 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] bulkstat_unlink_test 2024-03-16T18:02:09.530 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] deduperace 2024-03-16T18:02:09.904 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] bulkstat_unlink_test_modified 2024-03-16T18:02:10.281 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_dir_offset 2024-03-16T18:02:10.577 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_futimens 2024-03-16T18:02:10.882 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_immutable 2024-03-16T18:02:11.731 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] stale_handle 2024-03-16T18:02:12.055 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] pwrite_mmap_blocked 2024-03-16T18:02:12.376 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_dir_offset2 2024-03-16T18:02:12.742 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] seek_sanity_test 2024-03-16T18:02:13.384 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] seek_copy_test 2024-03-16T18:02:13.720 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_readdir_1 2024-03-16T18:02:14.064 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_readdir_2 2024-03-16T18:02:14.399 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fsync-tester 2024-03-16T18:02:14.824 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] nsexec 2024-03-16T18:02:15.162 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] cloner 2024-03-16T18:02:15.524 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] renameat2 2024-03-16T18:02:15.872 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_getcwd 2024-03-16T18:02:16.205 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] e4compact 2024-03-16T18:02:16.580 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] test-nextquota 2024-03-16T18:02:16.924 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] punch-alternating 2024-03-16T18:02:17.271 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] attr-list-by-handle-cursor-test 2024-03-16T18:02:17.620 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] listxattr 2024-03-16T18:02:17.935 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] dio-interleaved 2024-03-16T18:02:18.278 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_dir_type 2024-03-16T18:02:18.595 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] dio-invalidate-cache 2024-03-16T18:02:18.983 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] stat_test 2024-03-16T18:02:19.470 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_encrypted_d_revalidate 2024-03-16T18:02:19.817 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] attr_replace_test 2024-03-16T18:02:20.142 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] swapon 2024-03-16T18:02:20.476 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] mkswap 2024-03-16T18:02:20.768 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_attr_corruption 2024-03-16T18:02:21.092 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_open_tmpfiles 2024-03-16T18:02:21.456 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fscrypt-crypt-util 2024-03-16T18:02:22.314 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] bulkstat_null_ocount 2024-03-16T18:02:22.649 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] splice-test 2024-03-16T18:02:23.015 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] chprojid_fail 2024-03-16T18:02:23.344 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] detached_mounts_propagation 2024-03-16T18:02:23.682 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] ext4_resize 2024-03-16T18:02:24.014 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_readdir_3 2024-03-16T18:02:24.390 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] splice2pipe 2024-03-16T18:02:24.721 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] uuid_ioctl 2024-03-16T18:02:25.056 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_snapshot_deleted_subvolume 2024-03-16T18:02:25.398 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fiemap-tester 2024-03-16T18:02:25.823 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_stripealign 2024-03-16T18:02:26.156 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] open_by_handle 2024-03-16T18:02:26.607 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] dbtest 2024-03-16T18:02:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_mmap_dio 2024-03-16T18:02:27.381 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] xfsfind 2024-03-16T18:02:27.768 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] t_reflink_read_race 2024-03-16T18:02:28.149 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] fssum 2024-03-16T18:02:28.767 INFO:teuthology.orchestra.run.smithi082.stdout:Building log-writes 2024-03-16T18:02:28.778 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] replay-log 2024-03-16T18:02:29.216 INFO:teuthology.orchestra.run.smithi082.stdout:Building perf 2024-03-16T18:02:29.223 INFO:teuthology.orchestra.run.smithi082.stdout:Building aio-dio-regress 2024-03-16T18:02:29.234 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aiocp 2024-03-16T18:02:29.577 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-append-write-fallocate-race 2024-03-16T18:02:29.917 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-append-write-read-race 2024-03-16T18:02:30.260 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-cow-race 2024-03-16T18:02:30.596 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-cycle-write 2024-03-16T18:02:30.935 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-eof-race 2024-03-16T18:02:31.308 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-extend-stat 2024-03-16T18:02:31.658 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-fcntl-race 2024-03-16T18:02:31.981 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-hole-filling-race 2024-03-16T18:02:32.281 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-invalidate-failure 2024-03-16T18:02:32.628 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-invalidate-readahead 2024-03-16T18:02:32.971 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aiodio_sparse2 2024-03-16T18:02:33.367 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-subblock-eof-read 2024-03-16T18:02:33.682 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-dio-write-verify 2024-03-16T18:02:34.068 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-03-16T18:02:34.350 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-03-16T18:02:34.618 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] aio-last-ref-held-by-io 2024-03-16T18:02:34.931 INFO:teuthology.orchestra.run.smithi082.stdout:Building vfs 2024-03-16T18:02:34.944 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] vfstest 2024-03-16T18:02:36.945 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:02:36.953 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:02:38.898 INFO:teuthology.orchestra.run.smithi082.stdout: [CC] mount-idmapped 2024-03-16T18:02:39.457 INFO:teuthology.orchestra.run.smithi082.stdout:Building m4 2024-03-16T18:02:39.467 INFO:teuthology.orchestra.run.smithi082.stdout:Building common 2024-03-16T18:02:39.477 INFO:teuthology.orchestra.run.smithi082.stdout:Building tests 2024-03-16T18:02:39.492 INFO:teuthology.orchestra.run.smithi082.stdout:Building btrfs 2024-03-16T18:02:39.506 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/btrfs/group.list 2024-03-16T18:02:39.717 INFO:teuthology.orchestra.run.smithi082.stdout:Building ceph 2024-03-16T18:02:39.731 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/ceph/group.list 2024-03-16T18:02:39.785 INFO:teuthology.orchestra.run.smithi082.stdout:Building cifs 2024-03-16T18:02:39.799 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/cifs/group.list 2024-03-16T18:02:39.841 INFO:teuthology.orchestra.run.smithi082.stdout:Building ext4 2024-03-16T18:02:39.857 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/ext4/group.list 2024-03-16T18:02:40.000 INFO:teuthology.orchestra.run.smithi082.stdout:Building f2fs 2024-03-16T18:02:40.014 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/f2fs/group.list 2024-03-16T18:02:40.077 INFO:teuthology.orchestra.run.smithi082.stdout:Building generic 2024-03-16T18:02:40.095 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/generic/group.list 2024-03-16T18:02:40.457 INFO:teuthology.orchestra.run.smithi082.stdout:Building nfs 2024-03-16T18:02:40.472 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/nfs/group.list 2024-03-16T18:02:40.524 INFO:teuthology.orchestra.run.smithi082.stdout:Building ocfs2 2024-03-16T18:02:40.537 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/ocfs2/group.list 2024-03-16T18:02:40.585 INFO:teuthology.orchestra.run.smithi082.stdout:Building overlay 2024-03-16T18:02:40.599 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/overlay/group.list 2024-03-16T18:02:40.742 INFO:teuthology.orchestra.run.smithi082.stdout:Building perf 2024-03-16T18:02:40.756 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/perf/group.list 2024-03-16T18:02:40.795 INFO:teuthology.orchestra.run.smithi082.stdout:Building selftest 2024-03-16T18:02:40.808 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/selftest/group.list 2024-03-16T18:02:40.851 INFO:teuthology.orchestra.run.smithi082.stdout:Building shared 2024-03-16T18:02:40.865 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/shared/group.list 2024-03-16T18:02:40.929 INFO:teuthology.orchestra.run.smithi082.stdout:Building tmpfs 2024-03-16T18:02:40.944 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/tmpfs/group.list 2024-03-16T18:02:40.991 INFO:teuthology.orchestra.run.smithi082.stdout:Building udf 2024-03-16T18:02:41.005 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/udf/group.list 2024-03-16T18:02:41.042 INFO:teuthology.orchestra.run.smithi082.stdout:Building xfs 2024-03-16T18:02:41.058 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/xfs/group.list 2024-03-16T18:02:41.406 DEBUG:teuthology.orchestra.run.smithi082:> (cd /tmp/tmp.YEFrEDdeoexfstests-dev && exec sudo make install) 2024-03-16T18:02:41.527 INFO:teuthology.orchestra.run.smithi082.stdout:Building include 2024-03-16T18:02:41.536 INFO:teuthology.orchestra.run.smithi082.stdout:Building lib 2024-03-16T18:02:41.546 INFO:teuthology.orchestra.run.smithi082.stdout:Building ltp 2024-03-16T18:02:41.557 INFO:teuthology.orchestra.run.smithi082.stdout:Building src 2024-03-16T18:02:41.642 INFO:teuthology.orchestra.run.smithi082.stdout:Building log-writes 2024-03-16T18:02:41.652 INFO:teuthology.orchestra.run.smithi082.stdout:Building perf 2024-03-16T18:02:41.663 INFO:teuthology.orchestra.run.smithi082.stdout:Building aio-dio-regress 2024-03-16T18:02:41.675 INFO:teuthology.orchestra.run.smithi082.stdout:Building vfs 2024-03-16T18:02:41.686 INFO:teuthology.orchestra.run.smithi082.stdout:Building m4 2024-03-16T18:02:41.696 INFO:teuthology.orchestra.run.smithi082.stdout:Building common 2024-03-16T18:02:41.707 INFO:teuthology.orchestra.run.smithi082.stdout:Building tests 2024-03-16T18:02:41.721 INFO:teuthology.orchestra.run.smithi082.stdout:Building btrfs 2024-03-16T18:02:41.736 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/btrfs/group.list 2024-03-16T18:02:41.950 INFO:teuthology.orchestra.run.smithi082.stdout:Building ceph 2024-03-16T18:02:41.964 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/ceph/group.list 2024-03-16T18:02:42.013 INFO:teuthology.orchestra.run.smithi082.stdout:Building cifs 2024-03-16T18:02:42.027 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/cifs/group.list 2024-03-16T18:02:42.069 INFO:teuthology.orchestra.run.smithi082.stdout:Building ext4 2024-03-16T18:02:42.082 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/ext4/group.list 2024-03-16T18:02:42.219 INFO:teuthology.orchestra.run.smithi082.stdout:Building f2fs 2024-03-16T18:02:42.232 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/f2fs/group.list 2024-03-16T18:02:42.293 INFO:teuthology.orchestra.run.smithi082.stdout:Building generic 2024-03-16T18:02:42.311 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/generic/group.list 2024-03-16T18:02:42.643 INFO:teuthology.orchestra.run.smithi082.stdout:Building nfs 2024-03-16T18:02:42.658 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/nfs/group.list 2024-03-16T18:02:42.707 INFO:teuthology.orchestra.run.smithi082.stdout:Building ocfs2 2024-03-16T18:02:42.722 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/ocfs2/group.list 2024-03-16T18:02:42.769 INFO:teuthology.orchestra.run.smithi082.stdout:Building overlay 2024-03-16T18:02:42.784 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/overlay/group.list 2024-03-16T18:02:42.934 INFO:teuthology.orchestra.run.smithi082.stdout:Building perf 2024-03-16T18:02:42.948 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/perf/group.list 2024-03-16T18:02:42.987 INFO:teuthology.orchestra.run.smithi082.stdout:Building selftest 2024-03-16T18:02:43.001 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/selftest/group.list 2024-03-16T18:02:43.042 INFO:teuthology.orchestra.run.smithi082.stdout:Building shared 2024-03-16T18:02:43.056 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/shared/group.list 2024-03-16T18:02:43.119 INFO:teuthology.orchestra.run.smithi082.stdout:Building tmpfs 2024-03-16T18:02:43.132 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/tmpfs/group.list 2024-03-16T18:02:43.182 INFO:teuthology.orchestra.run.smithi082.stdout:Building udf 2024-03-16T18:02:43.196 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/udf/group.list 2024-03-16T18:02:43.236 INFO:teuthology.orchestra.run.smithi082.stdout:Building xfs 2024-03-16T18:02:43.254 INFO:teuthology.orchestra.run.smithi082.stdout: [GROUP] /tmp/tmp.YEFrEDdeoexfstests-dev/tests/xfs/group.list 2024-03-16T18:02:43.609 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-03-16T18:02:43.613 INFO:teuthology.orchestra.run.smithi082.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-16T18:02:43.613 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-03-16T18:02:43.618 INFO:teuthology.orchestra.run.smithi082.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-16T18:02:43.619 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-03-16T18:02:43.623 INFO:teuthology.orchestra.run.smithi082.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-03-16T18:02:43.656 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-03-16T18:02:43.885 INFO:teuthology.orchestra.run.smithi082.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-03-16T18:02:43.918 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-03-16T18:02:43.963 INFO:teuthology.orchestra.run.smithi082.stdout:Building log-writes 2024-03-16T18:02:43.973 INFO:teuthology.orchestra.run.smithi082.stdout:Building perf 2024-03-16T18:02:43.983 INFO:teuthology.orchestra.run.smithi082.stdout:Building aio-dio-regress 2024-03-16T18:02:43.994 INFO:teuthology.orchestra.run.smithi082.stdout:Building vfs 2024-03-16T18:02:44.002 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake -C log-writes install 2024-03-16T18:02:44.006 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-03-16T18:02:44.037 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-03-16T18:02:44.068 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake -C perf install 2024-03-16T18:02:44.071 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-03-16T18:02:44.102 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-03-16T18:02:44.178 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-03-16T18:02:44.182 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-03-16T18:02:44.213 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-03-16T18:02:44.395 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake -C vfs install 2024-03-16T18:02:44.399 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-03-16T18:02:44.431 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-03-16T18:02:44.473 INFO:teuthology.orchestra.run.smithi082.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-03-16T18:02:44.503 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-03-16T18:02:48.840 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-03-16T18:02:49.166 INFO:teuthology.orchestra.run.smithi082.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-03-16T18:02:49.227 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-03-16T18:02:49.230 INFO:teuthology.orchestra.run.smithi082.stdout:gmake[1]: Nothing to be done for 'install'. 2024-03-16T18:02:49.230 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-03-16T18:02:49.233 INFO:teuthology.orchestra.run.smithi082.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-03-16T18:02:49.265 INFO:teuthology.orchestra.run.smithi082.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-03-16T18:02:49.754 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-03-16T18:02:49.757 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/btrfs/ install 2024-03-16T18:02:49.764 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-03-16T18:02:49.794 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 304 305 306 307 308 309 310 /var/lib/xfstests/tests/btrfs 2024-03-16T18:02:52.947 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-03-16T18:02:52.977 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out /var/lib/xfstests/tests/btrfs 2024-03-16T18:02:56.132 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/ceph/ install 2024-03-16T18:02:56.136 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-03-16T18:02:56.169 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-03-16T18:02:56.240 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-03-16T18:02:56.271 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-03-16T18:02:56.339 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/cifs/ install 2024-03-16T18:02:56.343 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-03-16T18:02:56.374 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-03-16T18:02:56.405 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-03-16T18:02:56.437 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-03-16T18:02:56.469 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/ext4/ install 2024-03-16T18:02:56.474 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-03-16T18:02:56.506 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-03-16T18:02:57.189 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-03-16T18:02:57.220 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-03-16T18:02:58.007 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/f2fs/ install 2024-03-16T18:02:58.011 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-03-16T18:02:58.043 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-03-16T18:02:58.085 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-03-16T18:02:58.118 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-03-16T18:02:58.160 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/generic/ install 2024-03-16T18:02:58.171 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-03-16T18:02:58.204 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 /var/lib/xfstests/tests/generic 2024-03-16T18:03:06.583 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-03-16T18:03:06.619 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out /var/lib/xfstests/tests/generic 2024-03-16T18:03:07.095 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:03:07.101 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:03:14.495 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/nfs/ install 2024-03-16T18:03:14.499 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-03-16T18:03:14.530 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-03-16T18:03:14.560 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-03-16T18:03:14.591 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-03-16T18:03:14.622 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/ocfs2/ install 2024-03-16T18:03:14.626 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-03-16T18:03:14.657 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-03-16T18:03:14.687 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-03-16T18:03:14.717 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-03-16T18:03:14.748 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/overlay/ install 2024-03-16T18:03:14.751 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-03-16T18:03:14.781 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-03-16T18:03:15.880 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-03-16T18:03:15.909 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-03-16T18:03:16.994 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/perf/ install 2024-03-16T18:03:16.998 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-03-16T18:03:17.032 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-03-16T18:03:17.063 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-03-16T18:03:17.095 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-03-16T18:03:17.127 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/selftest/ install 2024-03-16T18:03:17.130 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-03-16T18:03:17.163 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-03-16T18:03:17.242 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-03-16T18:03:17.274 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-03-16T18:03:17.353 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/shared/ install 2024-03-16T18:03:17.356 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-03-16T18:03:17.390 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-03-16T18:03:17.441 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-03-16T18:03:17.468 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-03-16T18:03:17.518 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/tmpfs/ install 2024-03-16T18:03:17.522 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-03-16T18:03:17.555 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-03-16T18:03:17.586 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-03-16T18:03:17.616 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-03-16T18:03:17.647 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/udf/ install 2024-03-16T18:03:17.652 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-03-16T18:03:17.682 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-03-16T18:03:17.713 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-03-16T18:03:17.746 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-03-16T18:03:17.779 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.YEFrEDdeoexfstests-dev/tests/xfs/ install 2024-03-16T18:03:17.789 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-03-16T18:03:17.821 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-03-16T18:03:24.920 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-03-16T18:03:24.952 INFO:teuthology.orchestra.run.smithi082.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-03-16T18:03:32.695 INFO:teuthology.orchestra.run.smithi082.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-03-16T18:03:32.727 INFO:teuthology.orchestra.run.smithi082.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-03-16T18:03:32.759 INFO:teuthology.orchestra.run.smithi082.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-03-16T18:03:32.791 INFO:teuthology.orchestra.run.smithi082.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-03-16T18:03:32.828 INFO:tasks.cephfs.test_acls:client is kernel mounted 2024-03-16T18:03:32.828 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-16T18:03:32.829 DEBUG:teuthology.orchestra.run.smithi082:running tests for ACLs from xfstests-dev> (cd /tmp/tmp.YEFrEDdeoexfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-03-16T18:03:33.435 INFO:teuthology.orchestra.run.smithi082.stdout:FSTYP -- ceph 2024-03-16T18:03:33.449 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM -- Linux/x86_64 smithi082 6.8.0-rc7-g3845b7ac715a #1 SMP PREEMPT_DYNAMIC Thu Mar 14 10:49:05 UTC 2024 2024-03-16T18:03:33.450 INFO:teuthology.orchestra.run.smithi082.stdout:MKFS_OPTIONS -- 172.21.15.82:6789:/scratch 2024-03-16T18:03:33.454 INFO:teuthology.orchestra.run.smithi082.stdout:MOUNT_OPTIONS -- -o name=admin,secret=AQBW3fVlHzMbAhAAPrAs7qHZeUgs4cGOFpD7ow== -o context=system_u:object_r:root_t:s0 172.21.15.82:6789:/scratch /tmp/tmp.npBvz5wtGascratch 2024-03-16T18:03:33.454 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:35.939 INFO:teuthology.orchestra.run.smithi082.stdout:generic/099 2s 2024-03-16T18:03:35.957 INFO:teuthology.orchestra.run.smithi082.stdout:Ran: generic/099 2024-03-16T18:03:35.957 INFO:teuthology.orchestra.run.smithi082.stdout:Passed all 1 tests 2024-03-16T18:03:35.966 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:36.073 INFO:tasks.cephfs.test_acls:Command return value: 0 2024-03-16T18:03:36.073 INFO:tasks.cephfs.test_acls:Command stdout - FSTYP -- ceph PLATFORM -- Linux/x86_64 smithi082 6.8.0-rc7-g3845b7ac715a #1 SMP PREEMPT_DYNAMIC Thu Mar 14 10:49:05 UTC 2024 MKFS_OPTIONS -- 172.21.15.82:6789:/scratch MOUNT_OPTIONS -- -o name=admin,secret=AQBW3fVlHzMbAhAAPrAs7qHZeUgs4cGOFpD7ow== -o context=system_u:object_r:root_t:s0 172.21.15.82:6789:/scratch /tmp/tmp.npBvz5wtGascratch generic/099 2s Ran: generic/099 Passed all 1 tests 2024-03-16T18:03:36.073 INFO:tasks.cephfs.test_acls:Command stderr - 2024-03-16T18:03:36.073 DEBUG:teuthology.orchestra.run.smithi082:> sudo userdel --force --remove fsgqa 2024-03-16T18:03:36.745 DEBUG:teuthology.orchestra.run.smithi082:> sudo userdel --force --remove 123456-fsgqa 2024-03-16T18:03:37.228 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:03:37.237 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:03:37.378 DEBUG:teuthology.orchestra.run.smithi082:> sudo groupdel fsgqa 2024-03-16T18:03:37.527 INFO:teuthology.orchestra.run.smithi082.stderr:groupdel: group 'fsgqa' does not exist 2024-03-16T18:03:37.531 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-03-16T18:03:37.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:37.532 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.YEFrEDdeoexfstests-dev/results) 2024-03-16T18:03:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: File: /tmp/tmp.YEFrEDdeoexfstests-dev/results 2024-03-16T18:03:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 4096 Blocks: 8 IO Block: 4096 directory 2024-03-16T18:03:37.624 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 801h/2049d Inode: 684081 Links: 3 2024-03-16T18:03:37.624 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T18:03:37.624 INFO:teuthology.orchestra.run.smithi082.stdout:Context: unconfined_u:object_r:user_tmp_t:s0 2024-03-16T18:03:37.624 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-03-16 18:03:33.377189938 +0000 2024-03-16T18:03:37.624 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-03-16 18:03:35.950141076 +0000 2024-03-16T18:03:37.624 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-03-16 18:03:35.950141076 +0000 2024-03-16T18:03:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: 2024-03-16 18:03:33.377189938 +0000 2024-03-16T18:03:37.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:37.628 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p /var/log/ceph/xfstests-dev-results) 2024-03-16T18:03:37.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:37.721 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -r /tmp/tmp.YEFrEDdeoexfstests-dev/results /var/log/ceph/xfstests-dev-results) 2024-03-16T18:03:37.815 INFO:tasks.cephfs.xfstests_dev:results dir from xfstests-dev has been saved; it was copied from /tmp/tmp.YEFrEDdeoexfstests-dev to /var/log/ceph. 2024-03-16T18:03:37.815 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf /tmp/tmp.YEFrEDdeoexfstests-dev 2024-03-16T18:03:38.130 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-16T18:03:38.130 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T18:03:38.236 DEBUG:teuthology.orchestra.run.smithi090:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T18:03:38.346 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-16T18:03:38.346 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T18:03:38.346 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T18:03:38.404 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T18:03:38.404 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T18:03:38.459 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T18:03:38.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.666+0000 7f721a200640 1 Processor -- start 2024-03-16T18:03:38.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.667+0000 7f721a200640 1 -- start start 2024-03-16T18:03:38.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.668+0000 7f721a200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7214145320 0x7f7214165f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:38.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.668+0000 7f721a200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7214148950 con 0x7f7214166480 2024-03-16T18:03:38.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.668+0000 7f721a200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72140b3480 con 0x7f7214148120 2024-03-16T18:03:38.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.668+0000 7f721a200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7214166910 con 0x7f7214145320 2024-03-16T18:03:38.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.668+0000 7f720aa00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7214166480 0x7f7214147a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51848/0 (socket says 172.21.15.82:51848) 2024-03-16T18:03:38.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.668+0000 7f720aa00640 1 -- 172.21.15.82:0/328326360 learned_addr learned my addr 172.21.15.82:0/328326360 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:38.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7213400640 1 --2- 172.21.15.82:0/328326360 >> v2:172.21.15.90:3300/0 conn(0x7f7214145320 0x7f7214165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:38.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658236675 0 0) 0x7f7214148950 con 0x7f7214166480 2024-03-16T18:03:38.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71f8003520 con 0x7f7214166480 2024-03-16T18:03:38.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2360382988 0 0) 0x7f71f8003520 con 0x7f7214166480 2024-03-16T18:03:38.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 >> v2:172.21.15.90:3300/0 conn(0x7f7214145320 msgr2=0x7f7214165f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:38.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7212a00640 1 --2- 172.21.15.82:0/328326360 >> v2:172.21.15.90:3300/0 conn(0x7f7214145320 0x7f7214165f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 >> v1:172.21.15.90:6789/0 conn(0x7f7214148120 legacy=0x7f721416c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214145700 con 0x7f7214166480 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f71fc001670 con 0x7f7214166480 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 >> v1:172.21.15.82:6789/0 conn(0x7f7214166480 legacy=0x7f7214147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.669+0000 7f7212a00640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71f8003970 0x7f71f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f7212a00640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80047c0 0x7f71f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f7212a00640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8005180 0x7f71f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7214166910 con 0x7f71f80047c0 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f800c020 con 0x7f71f8005180 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f800c1e0 con 0x7f71f8003970 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f720aa00640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80047c0 0x7f71f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:38.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f7213e00640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8005180 0x7f71f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:38.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f7213400640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71f8003970 0x7f71f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:38.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f720aa00640 1 -- 172.21.15.82:0/328326360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71f8003970 msgr2=0x7f71f8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:38.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f720aa00640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f71f8003970 0x7f71f8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f720aa00640 1 -- 172.21.15.82:0/328326360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8005180 msgr2=0x7f71f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:38.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f720aa00640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8005180 0x7f71f800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f7213e00640 1 -- 172.21.15.82:0/328326360 reap_dead start 2024-03-16T18:03:38.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.670+0000 7f720aa00640 1 -- 172.21.15.82:0/328326360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f8003520 con 0x7f71f80047c0 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.672+0000 7f720aa00640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80047c0 0x7f71f8004c40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00c190 tx=0x7f71f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889e689b0d53e57c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.672+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc005ad0 con 0x7f71f80047c0 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.672+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc005c30 con 0x7f71f80047c0 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.672+0000 7f7212a00640 1 -- 172.21.15.82:0/328326360 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc02cae0 con 0x7f71f80047c0 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.672+0000 7f721a200640 1 -- 172.21.15.82:0/328326360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80047c0 msgr2=0x7f71f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.672+0000 7f721a200640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80047c0 0x7f71f8004c40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00c190 tx=0x7f71f80057f0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.672+0000 7f721a200640 1 -- 172.21.15.82:0/328326360 shutdown_connections 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.672+0000 7f721a200640 1 --2- 172.21.15.82:0/328326360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80047c0 0x7f71f8004c40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.672+0000 7f721a200640 1 -- 172.21.15.82:0/328326360 >> 172.21.15.82:0/328326360 conn(0x7f721406ae70 msgr2=0x7f721406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.673+0000 7f721a200640 1 -- 172.21.15.82:0/328326360 shutdown_connections 2024-03-16T18:03:38.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.673+0000 7f721a200640 1 -- 172.21.15.82:0/328326360 wait complete. 2024-03-16T18:03:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.674+0000 7f721a200640 1 Processor -- start 2024-03-16T18:03:38.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.675+0000 7f721a200640 1 -- start start 2024-03-16T18:03:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.675+0000 7f721a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f80047c0 0x7f72141f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.675+0000 7f721a200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7214143910 0x7f7214143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.675+0000 7f721a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7214164d70 0x7f72141651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:38.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.675+0000 7f721a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72141442d0 con 0x7f7214164d70 2024-03-16T18:03:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.675+0000 7f721a200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7214165c60 con 0x7f71f80047c0 2024-03-16T18:03:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.675+0000 7f721a200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7214165730 con 0x7f7214143910 2024-03-16T18:03:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7214164d70 0x7f72141651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7214164d70 0x7f72141651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35792/0 (socket says 172.21.15.82:35792) 2024-03-16T18:03:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213e00640 1 -- 172.21.15.82:0/2411532124 learned_addr learned my addr 172.21.15.82:0/2411532124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f720aa00640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7214143910 0x7f7214143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213400640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f80047c0 0x7f72141f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213e00640 1 -- 172.21.15.82:0/2411532124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7214143910 msgr2=0x7f7214143d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:38.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213e00640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7214143910 0x7f7214143d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213e00640 1 -- 172.21.15.82:0/2411532124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f80047c0 msgr2=0x7f72141f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213e00640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f80047c0 0x7f72141f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213e00640 1 -- 172.21.15.82:0/2411532124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214165980 con 0x7f7214164d70 2024-03-16T18:03:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f720aa00640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7214143910 0x7f7214143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213400640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f80047c0 0x7f72141f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.676+0000 7f7213e00640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7214164d70 0x7f72141651f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7214166860 tx=0x7f720c00e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.677+0000 7f720be00640 1 -- 172.21.15.82:0/2411532124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c006d60 con 0x7f7214164d70 2024-03-16T18:03:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.677+0000 7f720be00640 1 -- 172.21.15.82:0/2411532124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c003970 con 0x7f7214164d70 2024-03-16T18:03:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.677+0000 7f720be00640 1 -- 172.21.15.82:0/2411532124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c003d20 con 0x7f7214164d70 2024-03-16T18:03:38.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.677+0000 7f721a200640 1 -- 172.21.15.82:0/2411532124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72141f91e0 con 0x7f7214164d70 2024-03-16T18:03:38.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.677+0000 7f721a200640 1 -- 172.21.15.82:0/2411532124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72141f97a0 con 0x7f7214164d70 2024-03-16T18:03:38.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.678+0000 7f721a200640 1 -- 172.21.15.82:0/2411532124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72141f9030 con 0x7f7214164d70 2024-03-16T18:03:38.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.679+0000 7f720be00640 1 -- 172.21.15.82:0/2411532124 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f720c005110 con 0x7f7214164d70 2024-03-16T18:03:38.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.680+0000 7f720be00640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f71f006b3b0 0x7f71f006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:38.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.680+0000 7f720be00640 1 -- 172.21.15.82:0/2411532124 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f720c0893b0 con 0x7f7214164d70 2024-03-16T18:03:38.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.680+0000 7f7213400640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f71f006b3b0 0x7f71f006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:38.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.681+0000 7f7213400640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f71f006b3b0 0x7f71f006d870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7214163a10 tx=0x7f7204008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:38.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.685+0000 7f720be00640 1 -- 172.21.15.82:0/2411532124 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f720c054020 con 0x7f7214164d70 2024-03-16T18:03:38.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.880+0000 7f721a200640 1 -- 172.21.15.82:0/2411532124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72141f9990 con 0x7f7214164d70 2024-03-16T18:03:38.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.882+0000 7f720be00640 1 -- 172.21.15.82:0/2411532124 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16633 (secure 0 0 0) 0x7f720c0576e0 con 0x7f7214164d70 2024-03-16T18:03:38.886 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:38.887 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:51.442896+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:58:47.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T17:58:48.893361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 -- 172.21.15.82:0/2411532124 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f71f006b3b0 msgr2=0x7f71f006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f71f006b3b0 0x7f71f006d870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7214163a10 tx=0x7f7204008040 comp rx=0 tx=0).stop 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 -- 172.21.15.82:0/2411532124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7214164d70 msgr2=0x7f72141651f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7214164d70 0x7f72141651f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7214166860 tx=0x7f720c00e9e0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 -- 172.21.15.82:0/2411532124 shutdown_connections 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f71f006b3b0 0x7f71f006d870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7214164d70 0x7f72141651f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7214143910 0x7f7214143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 --2- 172.21.15.82:0/2411532124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f80047c0 0x7f72141f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 -- 172.21.15.82:0/2411532124 >> 172.21.15.82:0/2411532124 conn(0x7f721406ae70 msgr2=0x7f7214144a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 -- 172.21.15.82:0/2411532124 shutdown_connections 2024-03-16T18:03:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:38.885+0000 7f721a200640 1 -- 172.21.15.82:0/2411532124 wait complete. 2024-03-16T18:03:38.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:58:47.770770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:58:48.893361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T18:03:38.908 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T18:03:39.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.108+0000 7f2ca0a00640 1 Processor -- start 2024-03-16T18:03:39.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.108+0000 7f2ca0a00640 1 -- start start 2024-03-16T18:03:39.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.110+0000 7f2ca0a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2c9c06e940 0x7f2c9c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.110+0000 7f2ca0a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f2c9c0cc850 0x7f2c9c0cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.110+0000 7f2ca0a00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9c15c370 con 0x7f2c9c159960 2024-03-16T18:03:39.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.110+0000 7f2ca0a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c159d40 con 0x7f2c9c06e940 2024-03-16T18:03:39.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.110+0000 7f2ca0a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c15cff0 con 0x7f2c9c0cc850 2024-03-16T18:03:39.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.110+0000 7f2c99400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2c9c06e940 0x7f2c9c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.110+0000 7f2c99400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2c9c06e940 0x7f2c9c0cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35794/0 (socket says 172.21.15.82:35794) 2024-03-16T18:03:39.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.110+0000 7f2c99400640 1 -- 172.21.15.82:0/2689309501 learned_addr learned my addr 172.21.15.82:0/2689309501 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:39.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.111+0000 7f2c9a800640 1 --2- 172.21.15.82:0/2689309501 >> v2:172.21.15.90:3300/0 conn(0x7f2c9c0cc850 0x7f2c9c0cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.111+0000 7f2c99400640 1 -- 172.21.15.82:0/2689309501 >> v2:172.21.15.90:3300/0 conn(0x7f2c9c0cc850 msgr2=0x7f2c9c0cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:39.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.111+0000 7f2c99400640 1 --2- 172.21.15.82:0/2689309501 >> v2:172.21.15.90:3300/0 conn(0x7f2c9c0cc850 0x7f2c9c0cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.111+0000 7f2c99400640 1 -- 172.21.15.82:0/2689309501 >> v1:172.21.15.90:6789/0 conn(0x7f2c9c159960 legacy=0x7f2c9c06e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.111+0000 7f2c99400640 1 -- 172.21.15.82:0/2689309501 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c15c650 con 0x7f2c9c06e940 2024-03-16T18:03:39.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.112+0000 7f2c99400640 1 --2- 172.21.15.82:0/2689309501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 0x7f2c9c0cc310 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c15d380 tx=0x7f2c8802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c1c00e834c5d7e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:39.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.112+0000 7f2c98a00640 1 -- 172.21.15.82:0/2689309501 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88031870 con 0x7f2c9c06e940 2024-03-16T18:03:39.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.112+0000 7f2c98a00640 1 -- 172.21.15.82:0/2689309501 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c880319d0 con 0x7f2c9c06e940 2024-03-16T18:03:39.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.112+0000 7f2c98a00640 1 -- 172.21.15.82:0/2689309501 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c88031d60 con 0x7f2c9c06e940 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.113+0000 7f2ca0a00640 1 -- 172.21.15.82:0/2689309501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 msgr2=0x7f2c9c0cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.113+0000 7f2ca0a00640 1 --2- 172.21.15.82:0/2689309501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 0x7f2c9c0cc310 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c15d380 tx=0x7f2c8802b910 comp rx=0 tx=0).stop 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.113+0000 7f2ca0a00640 1 -- 172.21.15.82:0/2689309501 shutdown_connections 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.113+0000 7f2ca0a00640 1 --2- 172.21.15.82:0/2689309501 >> v2:172.21.15.90:3300/0 conn(0x7f2c9c0cc850 0x7f2c9c0cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.113+0000 7f2ca0a00640 1 --2- 172.21.15.82:0/2689309501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 0x7f2c9c0cc310 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.113+0000 7f2ca0a00640 1 -- 172.21.15.82:0/2689309501 >> 172.21.15.82:0/2689309501 conn(0x7f2c9c0b1640 msgr2=0x7f2c9c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.113+0000 7f2ca0a00640 1 -- 172.21.15.82:0/2689309501 shutdown_connections 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.113+0000 7f2ca0a00640 1 -- 172.21.15.82:0/2689309501 wait complete. 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.115+0000 7f2ca0a00640 1 Processor -- start 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.115+0000 7f2ca0a00640 1 -- start start 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.116+0000 7f2ca0a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 0x7f2c9c158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.116+0000 7f2ca0a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c9c0cc850 0x7f2c9c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.116+0000 7f2ca0a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2c9c159960 0x7f2c9c0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.116+0000 7f2ca0a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0c9f00 con 0x7f2c9c06e940 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.116+0000 7f2ca0a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0c8e60 con 0x7f2c9c0cc850 2024-03-16T18:03:39.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.116+0000 7f2ca0a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c0c9140 con 0x7f2c9c159960 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.116+0000 7f2c99e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 0x7f2c9c158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.116+0000 7f2c99e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 0x7f2c9c158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35798/0 (socket says 172.21.15.82:35798) 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.116+0000 7f2c99e00640 1 -- 172.21.15.82:0/1852191 learned_addr learned my addr 172.21.15.82:0/1852191 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2c9a800640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2c9c159960 0x7f2c9c0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2c99400640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c9c0cc850 0x7f2c9c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2c99e00640 1 -- 172.21.15.82:0/1852191 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2c9c159960 msgr2=0x7f2c9c0afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2c99e00640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2c9c159960 0x7f2c9c0afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2c99e00640 1 -- 172.21.15.82:0/1852191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c9c0cc850 msgr2=0x7f2c9c0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2c99e00640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c9c0cc850 0x7f2c9c0af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2c99e00640 1 -- 172.21.15.82:0/1852191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880094d0 con 0x7f2c9c06e940 2024-03-16T18:03:39.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2c99e00640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 0x7f2c9c158920 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00c3c0 tx=0x7f2c8c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:39.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2c92000640 1 -- 172.21.15.82:0/1852191 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c0052d0 con 0x7f2c9c06e940 2024-03-16T18:03:39.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2c92000640 1 -- 172.21.15.82:0/1852191 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c005450 con 0x7f2c9c06e940 2024-03-16T18:03:39.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.117+0000 7f2ca0a00640 1 -- 172.21.15.82:0/1852191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c0c9420 con 0x7f2c9c06e940 2024-03-16T18:03:39.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.118+0000 7f2c92000640 1 -- 172.21.15.82:0/1852191 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c007260 con 0x7f2c9c06e940 2024-03-16T18:03:39.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.118+0000 7f2ca0a00640 1 -- 172.21.15.82:0/1852191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c0c99e0 con 0x7f2c9c06e940 2024-03-16T18:03:39.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.119+0000 7f2ca0a00640 1 -- 172.21.15.82:0/1852191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c06c980 con 0x7f2c9c06e940 2024-03-16T18:03:39.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.120+0000 7f2c92000640 1 -- 172.21.15.82:0/1852191 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f2c8c0073c0 con 0x7f2c9c06e940 2024-03-16T18:03:39.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.121+0000 7f2c92000640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2c7c06b3e0 0x7f2c7c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.121+0000 7f2c92000640 1 -- 172.21.15.82:0/1852191 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c8c088fe0 con 0x7f2c9c06e940 2024-03-16T18:03:39.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.121+0000 7f2c99400640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2c7c06b3e0 0x7f2c7c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.122+0000 7f2c99400640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2c7c06b3e0 0x7f2c7c06d8a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2c88036000 tx=0x7f2c8803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:39.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.126+0000 7f2c92000640 1 -- 172.21.15.82:0/1852191 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2c9c06c980 con 0x7f2c9c06e940 2024-03-16T18:03:39.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.326+0000 7f2ca0a00640 1 -- 172.21.15.82:0/1852191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c9c06c980 con 0x7f2c9c06e940 2024-03-16T18:03:39.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.327+0000 7f2c92000640 1 -- 172.21.15.82:0/1852191 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c9c06c980 con 0x7f2c9c06e940 2024-03-16T18:03:39.331 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T18:03:39.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.330+0000 7f2ca0a00640 1 -- 172.21.15.82:0/1852191 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2c7c06b3e0 msgr2=0x7f2c7c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.330+0000 7f2ca0a00640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2c7c06b3e0 0x7f2c7c06d8a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2c88036000 tx=0x7f2c8803c040 comp rx=0 tx=0).stop 2024-03-16T18:03:39.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.330+0000 7f2ca0a00640 1 -- 172.21.15.82:0/1852191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 msgr2=0x7f2c9c158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.330+0000 7f2ca0a00640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 0x7f2c9c158920 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00c3c0 tx=0x7f2c8c00c880 comp rx=0 tx=0).stop 2024-03-16T18:03:39.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.330+0000 7f2ca0a00640 1 -- 172.21.15.82:0/1852191 shutdown_connections 2024-03-16T18:03:39.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.330+0000 7f2ca0a00640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2c7c06b3e0 0x7f2c7c06d8a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.330+0000 7f2ca0a00640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2c9c159960 0x7f2c9c0afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.330+0000 7f2ca0a00640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2c9c0cc850 0x7f2c9c0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.330+0000 7f2ca0a00640 1 --2- 172.21.15.82:0/1852191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2c9c06e940 0x7f2c9c158920 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.330+0000 7f2ca0a00640 1 -- 172.21.15.82:0/1852191 >> 172.21.15.82:0/1852191 conn(0x7f2c9c0b1640 msgr2=0x7f2c9c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:39.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.331+0000 7f2ca0a00640 1 -- 172.21.15.82:0/1852191 shutdown_connections 2024-03-16T18:03:39.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.331+0000 7f2ca0a00640 1 -- 172.21.15.82:0/1852191 wait complete. 2024-03-16T18:03:39.352 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T18:03:39.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.553+0000 7fdcf3200640 1 Processor -- start 2024-03-16T18:03:39.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.553+0000 7fdcf3200640 1 -- start start 2024-03-16T18:03:39.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.555+0000 7fdcf3200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fdcf41454d0 0x7fdcf4160280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.555+0000 7fdcf3200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdcf406e650 0x7fdcf40cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.555+0000 7fdcf3200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf41458b0 con 0x7fdcf406d900 2024-03-16T18:03:39.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.555+0000 7fdcf3200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf40cf3b0 con 0x7fdcf406e650 2024-03-16T18:03:39.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.555+0000 7fdcf3200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf40cf690 con 0x7fdcf41454d0 2024-03-16T18:03:39.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.555+0000 7fdce9400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdcf406e650 0x7fdcf40cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.555+0000 7fdce9400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdcf406e650 0x7fdcf40cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35802/0 (socket says 172.21.15.82:35802) 2024-03-16T18:03:39.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.555+0000 7fdce9400640 1 -- 172.21.15.82:0/3939017032 learned_addr learned my addr 172.21.15.82:0/3939017032 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:39.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.556+0000 7fdcf1e00640 1 --2- 172.21.15.82:0/3939017032 >> v2:172.21.15.90:3300/0 conn(0x7fdcf41454d0 0x7fdcf4160280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.556+0000 7fdce9400640 1 -- 172.21.15.82:0/3939017032 >> v2:172.21.15.90:3300/0 conn(0x7fdcf41454d0 msgr2=0x7fdcf4160280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:39.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.556+0000 7fdce9400640 1 --2- 172.21.15.82:0/3939017032 >> v2:172.21.15.90:3300/0 conn(0x7fdcf41454d0 0x7fdcf4160280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.556+0000 7fdce9400640 1 -- 172.21.15.82:0/3939017032 >> v1:172.21.15.90:6789/0 conn(0x7fdcf406d900 legacy=0x7fdcf40b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:39.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.556+0000 7fdce9400640 1 -- 172.21.15.82:0/3939017032 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf415f7a0 con 0x7fdcf406e650 2024-03-16T18:03:39.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.556+0000 7fdcf1e00640 1 --2- 172.21.15.82:0/3939017032 >> v2:172.21.15.90:3300/0 conn(0x7fdcf41454d0 0x7fdcf4160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:39.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.557+0000 7fdce9400640 1 --2- 172.21.15.82:0/3939017032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406e650 0x7fdcf40cd6f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdcf406c2a0 tx=0x7fdcdc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324913a3ee2d5049 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:39.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.557+0000 7fdcf1400640 1 -- 172.21.15.82:0/3939017032 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc031640 con 0x7fdcf406e650 2024-03-16T18:03:39.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.557+0000 7fdcf1400640 1 -- 172.21.15.82:0/3939017032 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc0317a0 con 0x7fdcf406e650 2024-03-16T18:03:39.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.557+0000 7fdcf1400640 1 -- 172.21.15.82:0/3939017032 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc031b30 con 0x7fdcf406e650 2024-03-16T18:03:39.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.558+0000 7fdcf3200640 1 -- 172.21.15.82:0/3939017032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406e650 msgr2=0x7fdcf40cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.558+0000 7fdcf3200640 1 --2- 172.21.15.82:0/3939017032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406e650 0x7fdcf40cd6f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdcf406c2a0 tx=0x7fdcdc02b8f0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.558+0000 7fdcf3200640 1 -- 172.21.15.82:0/3939017032 shutdown_connections 2024-03-16T18:03:39.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.558+0000 7fdcf3200640 1 --2- 172.21.15.82:0/3939017032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406e650 0x7fdcf40cd6f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.558+0000 7fdcf3200640 1 --2- 172.21.15.82:0/3939017032 >> v2:172.21.15.90:3300/0 conn(0x7fdcf41454d0 0x7fdcf4160280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.558+0000 7fdcf3200640 1 -- 172.21.15.82:0/3939017032 >> 172.21.15.82:0/3939017032 conn(0x7fdcf406ae70 msgr2=0x7fdcf406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:39.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.558+0000 7fdcf3200640 1 -- 172.21.15.82:0/3939017032 shutdown_connections 2024-03-16T18:03:39.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.558+0000 7fdcf3200640 1 -- 172.21.15.82:0/3939017032 wait complete. 2024-03-16T18:03:39.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.560+0000 7fdcf3200640 1 Processor -- start 2024-03-16T18:03:39.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.560+0000 7fdcf3200640 1 -- start start 2024-03-16T18:03:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.561+0000 7fdcf3200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406d900 0x7fdcf40c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.561+0000 7fdcf3200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf406e650 0x7fdcf415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.561+0000 7fdcf3200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf41454d0 0x7fdcf40ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.561+0000 7fdcf3200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40c4ac0 con 0x7fdcf406d900 2024-03-16T18:03:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.561+0000 7fdcf3200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40c4da0 con 0x7fdcf406e650 2024-03-16T18:03:39.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.561+0000 7fdcf3200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40c5080 con 0x7fdcf41454d0 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.561+0000 7fdcf1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406d900 0x7fdcf40c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.561+0000 7fdcf1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406d900 0x7fdcf40c5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35808/0 (socket says 172.21.15.82:35808) 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.561+0000 7fdcf1e00640 1 -- 172.21.15.82:0/85964386 learned_addr learned my addr 172.21.15.82:0/85964386 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdce9400640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf406e650 0x7fdcf415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdcf2800640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf41454d0 0x7fdcf40ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdcf1e00640 1 -- 172.21.15.82:0/85964386 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf41454d0 msgr2=0x7fdcf40ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdcf1e00640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf41454d0 0x7fdcf40ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdcf1e00640 1 -- 172.21.15.82:0/85964386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf406e650 msgr2=0x7fdcf415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdcf1e00640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf406e650 0x7fdcf415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdcf1e00640 1 -- 172.21.15.82:0/85964386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0094d0 con 0x7fdcf406d900 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdce9400640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf406e650 0x7fdcf415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T18:03:39.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdcf1e00640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406d900 0x7fdcf40c5b60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c7f0 tx=0x7fdce400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdceaa00640 1 -- 172.21.15.82:0/85964386 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4007820 con 0x7fdcf406d900 2024-03-16T18:03:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdceaa00640 1 -- 172.21.15.82:0/85964386 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce4007980 con 0x7fdcf406d900 2024-03-16T18:03:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.562+0000 7fdceaa00640 1 -- 172.21.15.82:0/85964386 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4007cf0 con 0x7fdcf406d900 2024-03-16T18:03:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.563+0000 7fdcf3200640 1 -- 172.21.15.82:0/85964386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf40c5360 con 0x7fdcf406d900 2024-03-16T18:03:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.563+0000 7fdcf3200640 1 -- 172.21.15.82:0/85964386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf40add90 con 0x7fdcf406d900 2024-03-16T18:03:39.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.564+0000 7fdcf3200640 1 -- 172.21.15.82:0/85964386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf404f250 con 0x7fdcf406d900 2024-03-16T18:03:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.565+0000 7fdceaa00640 1 -- 172.21.15.82:0/85964386 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fdce40113f0 con 0x7fdcf406d900 2024-03-16T18:03:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.566+0000 7fdceaa00640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdcd006b3b0 0x7fdcd006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.566+0000 7fdceaa00640 1 -- 172.21.15.82:0/85964386 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdce4089e10 con 0x7fdcf406d900 2024-03-16T18:03:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.566+0000 7fdce9400640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdcd006b3b0 0x7fdcd006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:39.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.566+0000 7fdce9400640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdcd006b3b0 0x7fdcd006d870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc036000 tx=0x7fdcdc03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:39.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.570+0000 7fdceaa00640 1 -- 172.21.15.82:0/85964386 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdce4054940 con 0x7fdcf406d900 2024-03-16T18:03:39.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.764+0000 7fdcf3200640 1 -- 172.21.15.82:0/85964386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdcf406e430 con 0x7fdcf406d900 2024-03-16T18:03:39.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.765+0000 7fdceaa00640 1 -- 172.21.15.82:0/85964386 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdce401f0a0 con 0x7fdcf406d900 2024-03-16T18:03:39.768 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T18:03:39.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 -- 172.21.15.82:0/85964386 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdcd006b3b0 msgr2=0x7fdcd006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdcd006b3b0 0x7fdcd006d870 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc036000 tx=0x7fdcdc03c040 comp rx=0 tx=0).stop 2024-03-16T18:03:39.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 -- 172.21.15.82:0/85964386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406d900 msgr2=0x7fdcf40c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:39.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406d900 0x7fdcf40c5b60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c7f0 tx=0x7fdce400ccb0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 -- 172.21.15.82:0/85964386 shutdown_connections 2024-03-16T18:03:39.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdcd006b3b0 0x7fdcd006d870 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdcf41454d0 0x7fdcf40ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdcf406e650 0x7fdcf415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 --2- 172.21.15.82:0/85964386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcf406d900 0x7fdcf40c5b60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:39.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 -- 172.21.15.82:0/85964386 >> 172.21.15.82:0/85964386 conn(0x7fdcf406ae70 msgr2=0x7fdcf415c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:39.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 -- 172.21.15.82:0/85964386 shutdown_connections 2024-03-16T18:03:39.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.767+0000 7fdcf3200640 1 -- 172.21.15.82:0/85964386 wait complete. 2024-03-16T18:03:39.790 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T18:03:40.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.998+0000 7fc8eec00640 1 Processor -- start 2024-03-16T18:03:40.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.998+0000 7fc8eec00640 1 -- start start 2024-03-16T18:03:40.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.999+0000 7fc8eec00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8e80cf390 0x7fc8e80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.999+0000 7fc8eec00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8e80d28c0 0x7fc8e80b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.999+0000 7fc8eec00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e80cfc80 con 0x7fc8e815e0d0 2024-03-16T18:03:40.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.999+0000 7fc8eec00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e80d2d30 con 0x7fc8e80cf390 2024-03-16T18:03:40.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.999+0000 7fc8eec00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80cf9a0 con 0x7fc8e80d28c0 2024-03-16T18:03:40.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:39.999+0000 7fc8e7e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc8e815e0d0 0x7fc8e80d1c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51860/0 (socket says 172.21.15.82:51860) 2024-03-16T18:03:40.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.000+0000 7fc8e7e00640 1 -- 172.21.15.82:0/2476707184 learned_addr learned my addr 172.21.15.82:0/2476707184 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:40.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.000+0000 7fc8e7400640 1 --2- 172.21.15.82:0/2476707184 >> v2:172.21.15.82:3300/0 conn(0x7fc8e80cf390 0x7fc8e80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.000+0000 7fc8ecc00640 1 --2- 172.21.15.82:0/2476707184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8e80d28c0 0x7fc8e80b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.000+0000 7fc8e6a00640 1 -- 172.21.15.82:0/2476707184 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119398282 0 0) 0x7fc8e80cfc80 con 0x7fc8e815e0d0 2024-03-16T18:03:40.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.000+0000 7fc8e6a00640 1 -- 172.21.15.82:0/2476707184 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8cc003520 con 0x7fc8e815e0d0 2024-03-16T18:03:40.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.000+0000 7fc8e7400640 1 -- 172.21.15.82:0/2476707184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8e80d28c0 msgr2=0x7fc8e80b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:40.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.000+0000 7fc8e7400640 1 --2- 172.21.15.82:0/2476707184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8e80d28c0 0x7fc8e80b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.000+0000 7fc8e7400640 1 -- 172.21.15.82:0/2476707184 >> v1:172.21.15.82:6789/0 conn(0x7fc8e815e0d0 legacy=0x7fc8e80d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.000+0000 7fc8e7400640 1 -- 172.21.15.82:0/2476707184 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e80b37c0 con 0x7fc8e80cf390 2024-03-16T18:03:40.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.001+0000 7fc8e7400640 1 --2- 172.21.15.82:0/2476707184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80cf390 0x7fc8e80d2380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00097a0 tx=0x7fc8d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd9f6656a18f32d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:40.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8e6a00640 1 -- 172.21.15.82:0/2476707184 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0031830 con 0x7fc8e80cf390 2024-03-16T18:03:40.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8e6a00640 1 -- 172.21.15.82:0/2476707184 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d0031990 con 0x7fc8e80cf390 2024-03-16T18:03:40.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8e6a00640 1 -- 172.21.15.82:0/2476707184 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0031d20 con 0x7fc8e80cf390 2024-03-16T18:03:40.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8eec00640 1 -- 172.21.15.82:0/2476707184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80cf390 msgr2=0x7fc8e80d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8eec00640 1 --2- 172.21.15.82:0/2476707184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80cf390 0x7fc8e80d2380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00097a0 tx=0x7fc8d002b8d0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8eec00640 1 -- 172.21.15.82:0/2476707184 shutdown_connections 2024-03-16T18:03:40.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8eec00640 1 --2- 172.21.15.82:0/2476707184 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8e80d28c0 0x7fc8e80b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8eec00640 1 --2- 172.21.15.82:0/2476707184 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80cf390 0x7fc8e80d2380 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8eec00640 1 -- 172.21.15.82:0/2476707184 >> 172.21.15.82:0/2476707184 conn(0x7fc8e80d3dc0 msgr2=0x7fc8e80d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:40.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8eec00640 1 -- 172.21.15.82:0/2476707184 shutdown_connections 2024-03-16T18:03:40.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.002+0000 7fc8eec00640 1 -- 172.21.15.82:0/2476707184 wait complete. 2024-03-16T18:03:40.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.004+0000 7fc8eec00640 1 Processor -- start 2024-03-16T18:03:40.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.004+0000 7fc8eec00640 1 -- start start 2024-03-16T18:03:40.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.005+0000 7fc8eec00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e80cf390 0x7fc8e8155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.005+0000 7fc8eec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80d28c0 0x7fc8e80cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.005+0000 7fc8eec00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8e815e0d0 0x7fc8e81f3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.005+0000 7fc8eec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80cc290 con 0x7fc8e80d28c0 2024-03-16T18:03:40.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.005+0000 7fc8eec00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80c8eb0 con 0x7fc8e80cf390 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.005+0000 7fc8eec00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80cc4a0 con 0x7fc8e815e0d0 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.005+0000 7fc8e7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80d28c0 0x7fc8e80cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.005+0000 7fc8e7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80d28c0 0x7fc8e80cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35818/0 (socket says 172.21.15.82:35818) 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.006+0000 7fc8e7400640 1 -- 172.21.15.82:0/3086825000 learned_addr learned my addr 172.21.15.82:0/3086825000 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.006+0000 7fc8e7e00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e80cf390 0x7fc8e8155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.006+0000 7fc8ecc00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8e815e0d0 0x7fc8e81f3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.006+0000 7fc8e7400640 1 -- 172.21.15.82:0/3086825000 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8e815e0d0 msgr2=0x7fc8e81f3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.006+0000 7fc8e7400640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8e815e0d0 0x7fc8e81f3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.006+0000 7fc8e7400640 1 -- 172.21.15.82:0/3086825000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e80cf390 msgr2=0x7fc8e8155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.006+0000 7fc8e7400640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e80cf390 0x7fc8e8155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.006+0000 7fc8e7400640 1 -- 172.21.15.82:0/3086825000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d0009400 con 0x7fc8e80d28c0 2024-03-16T18:03:40.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.006+0000 7fc8e7400640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80d28c0 0x7fc8e80cd540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc8d002bee0 tx=0x7fc8d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:40.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.007+0000 7fc8dfe00640 1 -- 172.21.15.82:0/3086825000 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d003a6f0 con 0x7fc8e80d28c0 2024-03-16T18:03:40.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.007+0000 7fc8dfe00640 1 -- 172.21.15.82:0/3086825000 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d003a870 con 0x7fc8e80d28c0 2024-03-16T18:03:40.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.007+0000 7fc8dfe00640 1 -- 172.21.15.82:0/3086825000 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d003ac20 con 0x7fc8e80d28c0 2024-03-16T18:03:40.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.007+0000 7fc8eec00640 1 -- 172.21.15.82:0/3086825000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e80cc780 con 0x7fc8e80d28c0 2024-03-16T18:03:40.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.007+0000 7fc8eec00640 1 -- 172.21.15.82:0/3086825000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e80ccd40 con 0x7fc8e80d28c0 2024-03-16T18:03:40.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.008+0000 7fc8eec00640 1 -- 172.21.15.82:0/3086825000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e8068640 con 0x7fc8e80d28c0 2024-03-16T18:03:40.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.009+0000 7fc8dfe00640 1 -- 172.21.15.82:0/3086825000 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fc8d0030380 con 0x7fc8e80d28c0 2024-03-16T18:03:40.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.010+0000 7fc8dfe00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8c406b3b0 0x7fc8c406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.010+0000 7fc8dfe00640 1 -- 172.21.15.82:0/3086825000 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8d00ab640 con 0x7fc8e80d28c0 2024-03-16T18:03:40.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.010+0000 7fc8e7e00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8c406b3b0 0x7fc8c406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.011+0000 7fc8e7e00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8c406b3b0 0x7fc8c406d870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8005880 tx=0x7fc8d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:40.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.014+0000 7fc8dfe00640 1 -- 172.21.15.82:0/3086825000 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc8d0076200 con 0x7fc8e80d28c0 2024-03-16T18:03:40.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.212+0000 7fc8eec00640 1 -- 172.21.15.82:0/3086825000 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8e80b4470 con 0x7fc8e80d28c0 2024-03-16T18:03:40.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.213+0000 7fc8dfe00640 1 -- 172.21.15.82:0/3086825000 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8d00798c0 con 0x7fc8e80d28c0 2024-03-16T18:03:40.216 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 -- 172.21.15.82:0/3086825000 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8c406b3b0 msgr2=0x7fc8c406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8c406b3b0 0x7fc8c406d870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8005880 tx=0x7fc8d8009040 comp rx=0 tx=0).stop 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 -- 172.21.15.82:0/3086825000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80d28c0 msgr2=0x7fc8e80cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80d28c0 0x7fc8e80cd540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc8d002bee0 tx=0x7fc8d00046c0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 -- 172.21.15.82:0/3086825000 shutdown_connections 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fc8c406b3b0 0x7fc8c406d870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fc8e815e0d0 0x7fc8e81f3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8e80d28c0 0x7fc8e80cd540 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 --2- 172.21.15.82:0/3086825000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8e80cf390 0x7fc8e8155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 -- 172.21.15.82:0/3086825000 >> 172.21.15.82:0/3086825000 conn(0x7fc8e80d3dc0 msgr2=0x7fc8e80d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 -- 172.21.15.82:0/3086825000 shutdown_connections 2024-03-16T18:03:40.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.216+0000 7fc8eec00640 1 -- 172.21.15.82:0/3086825000 wait complete. 2024-03-16T18:03:40.239 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T18:03:40.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.452+0000 7f94a3a00640 1 Processor -- start 2024-03-16T18:03:40.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.452+0000 7f94a3a00640 1 -- start start 2024-03-16T18:03:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.454+0000 7f94a3a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f949c06c870 0x7f949c15f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.454+0000 7f94a3a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f949c06d410 0x7f949c06cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.454+0000 7f94a3a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f949c0ce820 0x7f949c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.454+0000 7f94a3a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c15f570 con 0x7f949c06d410 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.454+0000 7f94a3a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c0cddd0 con 0x7f949c06c870 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.454+0000 7f94a3a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c15bf80 con 0x7f949c0ce820 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.455+0000 7f949be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f949c06d410 0x7f949c06cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.455+0000 7f949be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f949c06d410 0x7f949c06cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35826/0 (socket says 172.21.15.82:35826) 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.455+0000 7f949be00640 1 -- 172.21.15.82:0/1665403113 learned_addr learned my addr 172.21.15.82:0/1665403113 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.455+0000 7f949be00640 1 -- 172.21.15.82:0/1665403113 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f949c0ce820 msgr2=0x7f949c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.455+0000 7f949be00640 1 --2- 172.21.15.82:0/1665403113 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f949c0ce820 0x7f949c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.455+0000 7f94a1000640 1 --2- 172.21.15.82:0/1665403113 >> v2:172.21.15.90:3300/0 conn(0x7f949c06c870 0x7f949c15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.455+0000 7f949be00640 1 -- 172.21.15.82:0/1665403113 >> v2:172.21.15.90:3300/0 conn(0x7f949c06c870 msgr2=0x7f949c15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.455+0000 7f949be00640 1 --2- 172.21.15.82:0/1665403113 >> v2:172.21.15.90:3300/0 conn(0x7f949c06c870 0x7f949c15f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.455+0000 7f949be00640 1 -- 172.21.15.82:0/1665403113 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c06d190 con 0x7f949c06d410 2024-03-16T18:03:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.456+0000 7f949be00640 1 --2- 172.21.15.82:0/1665403113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06d410 0x7f949c06cc50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f949c06ec60 tx=0x7f948402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808330acc7d6a6ba server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f949b400640 1 -- 172.21.15.82:0/1665403113 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484031860 con 0x7f949c06d410 2024-03-16T18:03:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f949b400640 1 -- 172.21.15.82:0/1665403113 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94840319c0 con 0x7f949c06d410 2024-03-16T18:03:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f949b400640 1 -- 172.21.15.82:0/1665403113 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484031d50 con 0x7f949c06d410 2024-03-16T18:03:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f94a3a00640 1 -- 172.21.15.82:0/1665403113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06d410 msgr2=0x7f949c06cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f94a3a00640 1 --2- 172.21.15.82:0/1665403113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06d410 0x7f949c06cc50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f949c06ec60 tx=0x7f948402b900 comp rx=0 tx=0).stop 2024-03-16T18:03:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f94a3a00640 1 -- 172.21.15.82:0/1665403113 shutdown_connections 2024-03-16T18:03:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f94a3a00640 1 --2- 172.21.15.82:0/1665403113 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f949c0ce820 0x7f949c0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f94a3a00640 1 --2- 172.21.15.82:0/1665403113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06d410 0x7f949c06cc50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f94a3a00640 1 --2- 172.21.15.82:0/1665403113 >> v2:172.21.15.90:3300/0 conn(0x7f949c06c870 0x7f949c15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f94a3a00640 1 -- 172.21.15.82:0/1665403113 >> 172.21.15.82:0/1665403113 conn(0x7f949c0b17b0 msgr2=0x7f949c0b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:40.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f94a3a00640 1 -- 172.21.15.82:0/1665403113 shutdown_connections 2024-03-16T18:03:40.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.457+0000 7f94a3a00640 1 -- 172.21.15.82:0/1665403113 wait complete. 2024-03-16T18:03:40.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.459+0000 7f94a3a00640 1 Processor -- start 2024-03-16T18:03:40.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.459+0000 7f94a3a00640 1 -- start start 2024-03-16T18:03:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.460+0000 7f94a3a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06c870 0x7f949c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.460+0000 7f94a3a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f949c06d410 0x7f949c149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.460+0000 7f94a3a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f949c0ce820 0x7f949c14a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.460+0000 7f94a3a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c157260 con 0x7f949c06c870 2024-03-16T18:03:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.460+0000 7f94a3a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c156f80 con 0x7f949c0ce820 2024-03-16T18:03:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.460+0000 7f94a3a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c1607a0 con 0x7f949c06d410 2024-03-16T18:03:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.460+0000 7f94a1000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06c870 0x7f949c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f94a1000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06c870 0x7f949c0b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35830/0 (socket says 172.21.15.82:35830) 2024-03-16T18:03:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f94a1000640 1 -- 172.21.15.82:0/3159630895 learned_addr learned my addr 172.21.15.82:0/3159630895 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f949be00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f949c06d410 0x7f949c149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f94a1a00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f949c0ce820 0x7f949c14a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f94a1000640 1 -- 172.21.15.82:0/3159630895 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f949c06d410 msgr2=0x7f949c149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f94a1000640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f949c06d410 0x7f949c149ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f94a1000640 1 -- 172.21.15.82:0/3159630895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f949c0ce820 msgr2=0x7f949c14a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f94a1000640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f949c0ce820 0x7f949c14a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f94a1000640 1 -- 172.21.15.82:0/3159630895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840094d0 con 0x7f949c06c870 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f94a1000640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06c870 0x7f949c0b2fc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c7f0 tx=0x7f948c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.461+0000 7f9498c00640 1 -- 172.21.15.82:0/3159630895 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c004220 con 0x7f949c06c870 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.462+0000 7f9498c00640 1 -- 172.21.15.82:0/3159630895 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c004380 con 0x7f949c06c870 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.462+0000 7f9498c00640 1 -- 172.21.15.82:0/3159630895 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c007840 con 0x7f949c06c870 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.462+0000 7f94a3a00640 1 -- 172.21.15.82:0/3159630895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c160a80 con 0x7f949c06c870 2024-03-16T18:03:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.462+0000 7f94a3a00640 1 -- 172.21.15.82:0/3159630895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c0b3500 con 0x7f949c06c870 2024-03-16T18:03:40.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.463+0000 7f94a3a00640 1 -- 172.21.15.82:0/3159630895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c0b36f0 con 0x7f949c06c870 2024-03-16T18:03:40.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.464+0000 7f9498c00640 1 -- 172.21.15.82:0/3159630895 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f948c007bb0 con 0x7f949c06c870 2024-03-16T18:03:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.465+0000 7f9498c00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f947806b3e0 0x7f947806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.465+0000 7f9498c00640 1 -- 172.21.15.82:0/3159630895 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f948c089b80 con 0x7f949c06c870 2024-03-16T18:03:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.465+0000 7f949be00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f947806b3e0 0x7f947806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.465+0000 7f949be00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f947806b3e0 0x7f947806d8a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9484036000 tx=0x7f948403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:40.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.469+0000 7f9498c00640 1 -- 172.21.15.82:0/3159630895 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f948c054740 con 0x7f949c06c870 2024-03-16T18:03:40.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.678+0000 7f94a3a00640 1 -- 172.21.15.82:0/3159630895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f949c160fa0 con 0x7f949c06c870 2024-03-16T18:03:40.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.679+0000 7f9498c00640 1 -- 172.21.15.82:0/3159630895 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) v1 ==== 80+0+0 (secure 0 0 0) 0x7f949c160fa0 con 0x7f949c06c870 2024-03-16T18:03:40.683 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-03-16T18:03:40.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 -- 172.21.15.82:0/3159630895 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f947806b3e0 msgr2=0x7f947806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f947806b3e0 0x7f947806d8a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9484036000 tx=0x7f948403c040 comp rx=0 tx=0).stop 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 -- 172.21.15.82:0/3159630895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06c870 msgr2=0x7f949c0b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06c870 0x7f949c0b2fc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c7f0 tx=0x7f948c00ccb0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 -- 172.21.15.82:0/3159630895 shutdown_connections 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f947806b3e0 0x7f947806d8a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f949c0ce820 0x7f949c14a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f949c06d410 0x7f949c149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 --2- 172.21.15.82:0/3159630895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f949c06c870 0x7f949c0b2fc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 -- 172.21.15.82:0/3159630895 >> 172.21.15.82:0/3159630895 conn(0x7f949c0b17b0 msgr2=0x7f949c0cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 -- 172.21.15.82:0/3159630895 shutdown_connections 2024-03-16T18:03:40.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:40.682+0000 7f94a3a00640 1 -- 172.21.15.82:0/3159630895 wait complete. 2024-03-16T18:03:40.704 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-16T18:03:40.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:40.704 DEBUG:teuthology.orchestra.run.smithi082:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-16T18:03:40.818 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T18:03:40.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:40.818 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-16T18:03:40.869 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T18:03:40.869 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T18:03:40.926 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-16T18:03:40.926 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T18:03:40.927 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T18:03:40.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-16T18:03:40.982 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T18:03:40.982 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T18:03:41.036 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-16T18:03:41.036 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T18:03:41.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.249+0000 7f4065000640 1 Processor -- start 2024-03-16T18:03:41.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.249+0000 7f4065000640 1 -- start start 2024-03-16T18:03:41.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.251+0000 7f4065000640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f40600b4430 0x7f406006c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.251+0000 7f4065000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406006d2e0 con 0x7f406015e180 2024-03-16T18:03:41.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.251+0000 7f4065000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406015e560 con 0x7f406006ced0 2024-03-16T18:03:41.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.251+0000 7f4065000640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40600d01b0 con 0x7f40600b4430 2024-03-16T18:03:41.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.251+0000 7f405de00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f406015e180 0x7f406006c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51874/0 (socket says 172.21.15.82:51874) 2024-03-16T18:03:41.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.251+0000 7f405de00640 1 -- 172.21.15.82:0/1577927399 learned_addr learned my addr 172.21.15.82:0/1577927399 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f4055400640 1 --2- 172.21.15.82:0/1577927399 >> v2:172.21.15.90:3300/0 conn(0x7f40600b4430 0x7f406006c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613457742 0 0) 0x7f406006d2e0 con 0x7f406015e180 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4044003520 con 0x7f406015e180 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3898584787 0 0) 0x7f4044003520 con 0x7f406015e180 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 >> v2:172.21.15.90:3300/0 conn(0x7f40600b4430 msgr2=0x7f406006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 --2- 172.21.15.82:0/1577927399 >> v2:172.21.15.90:3300/0 conn(0x7f40600b4430 0x7f406006c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 >> v1:172.21.15.90:6789/0 conn(0x7f406006ced0 legacy=0x7f40600cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40600d0490 con 0x7f406015e180 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f4050001670 con 0x7f406015e180 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 >> v1:172.21.15.82:6789/0 conn(0x7f406015e180 legacy=0x7f406006c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4044003970 0x7f4044003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40440047c0 0x7f4044004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4044005180 0x7f404400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40600d01b0 con 0x7f4044003970 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404400c0b0 con 0x7f4044005180 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404400c270 con 0x7f40440047c0 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.252+0000 7f405de00640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4044003970 0x7f4044003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.253+0000 7f405de00640 1 -- 172.21.15.82:0/1577927399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40440047c0 msgr2=0x7f4044004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.253+0000 7f4055400640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40440047c0 0x7f4044004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.253+0000 7f405e800640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4044005180 0x7f404400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.253+0000 7f405de00640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f40440047c0 0x7f4044004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.253+0000 7f405de00640 1 -- 172.21.15.82:0/1577927399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4044005180 msgr2=0x7f404400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.253+0000 7f405de00640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4044005180 0x7f404400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.253+0000 7f405de00640 1 -- 172.21.15.82:0/1577927399 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4044003520 con 0x7f4044003970 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.253+0000 7f405e800640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4044005180 0x7f404400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:41.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.253+0000 7f405e800640 1 -- 172.21.15.82:0/1577927399 reap_dead start 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.254+0000 7f405de00640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4044003970 0x7f4044003d70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4050001eb0 tx=0x7f40440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c954511a5b945b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.255+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050005ad0 con 0x7f4044003970 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.255+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4050005c50 con 0x7f4044003970 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.255+0000 7f405d400640 1 -- 172.21.15.82:0/1577927399 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405002c8b0 con 0x7f4044003970 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.256+0000 7f4065000640 1 -- 172.21.15.82:0/1577927399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4044003970 msgr2=0x7f4044003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.256+0000 7f4065000640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4044003970 0x7f4044003d70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4050001eb0 tx=0x7f40440057f0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.256+0000 7f4065000640 1 -- 172.21.15.82:0/1577927399 shutdown_connections 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.256+0000 7f4065000640 1 --2- 172.21.15.82:0/1577927399 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4044003970 0x7f4044003d70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.256+0000 7f4065000640 1 -- 172.21.15.82:0/1577927399 >> 172.21.15.82:0/1577927399 conn(0x7f40600b37f0 msgr2=0x7f40600b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.256+0000 7f4065000640 1 -- 172.21.15.82:0/1577927399 shutdown_connections 2024-03-16T18:03:41.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.256+0000 7f4065000640 1 -- 172.21.15.82:0/1577927399 wait complete. 2024-03-16T18:03:41.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.258+0000 7f4065000640 1 Processor -- start 2024-03-16T18:03:41.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.258+0000 7f4065000640 1 -- start start 2024-03-16T18:03:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.259+0000 7f4065000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4044003970 0x7f40600c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.259+0000 7f4065000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f406009bae0 0x7f40600b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.259+0000 7f4065000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40600ca860 0x7f406009ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.259+0000 7f4065000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406009c330 con 0x7f406009bae0 2024-03-16T18:03:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.259+0000 7f4065000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406009c050 con 0x7f40600ca860 2024-03-16T18:03:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.259+0000 7f4065000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406009e490 con 0x7f4044003970 2024-03-16T18:03:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.259+0000 7f4055400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f406009bae0 0x7f40600b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.259+0000 7f4055400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f406009bae0 0x7f40600b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35848/0 (socket says 172.21.15.82:35848) 2024-03-16T18:03:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.259+0000 7f4055400640 1 -- 172.21.15.82:0/2599647309 learned_addr learned my addr 172.21.15.82:0/2599647309 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:41.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f405de00640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4044003970 0x7f40600c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f405e800640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40600ca860 0x7f406009ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f4055400640 1 -- 172.21.15.82:0/2599647309 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4044003970 msgr2=0x7f40600c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f4055400640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4044003970 0x7f40600c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f4055400640 1 -- 172.21.15.82:0/2599647309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40600ca860 msgr2=0x7f406009ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f4055400640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40600ca860 0x7f406009ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f4055400640 1 -- 172.21.15.82:0/2599647309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406009e770 con 0x7f406009bae0 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f4055400640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f406009bae0 0x7f40600b6d20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f404800e7e0 tx=0x7f404800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f4056a00640 1 -- 172.21.15.82:0/2599647309 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800c6d0 con 0x7f406009bae0 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f4056a00640 1 -- 172.21.15.82:0/2599647309 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404800c830 con 0x7f406009bae0 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f4056a00640 1 -- 172.21.15.82:0/2599647309 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404800cba0 con 0x7f406009bae0 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.260+0000 7f4065000640 1 -- 172.21.15.82:0/2599647309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406009ea50 con 0x7f406009bae0 2024-03-16T18:03:41.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.261+0000 7f4065000640 1 -- 172.21.15.82:0/2599647309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40600b7260 con 0x7f406009bae0 2024-03-16T18:03:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.262+0000 7f4065000640 1 -- 172.21.15.82:0/2599647309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406009e2f0 con 0x7f406009bae0 2024-03-16T18:03:41.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.263+0000 7f4056a00640 1 -- 172.21.15.82:0/2599647309 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f404800cd00 con 0x7f406009bae0 2024-03-16T18:03:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.263+0000 7f4056a00640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f403c06b3b0 0x7f403c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.264+0000 7f4056a00640 1 -- 172.21.15.82:0/2599647309 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4048089910 con 0x7f406009bae0 2024-03-16T18:03:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.264+0000 7f405de00640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f403c06b3b0 0x7f403c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.264+0000 7f405de00640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f403c06b3b0 0x7f403c06d870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4050034000 tx=0x7f4050009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:41.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.268+0000 7f4056a00640 1 -- 172.21.15.82:0/2599647309 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f406009e2f0 con 0x7f406009bae0 2024-03-16T18:03:41.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.500+0000 7f4065000640 1 -- 172.21.15.82:0/2599647309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f406009e2f0 con 0x7f406009bae0 2024-03-16T18:03:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.501+0000 7f4056a00640 1 -- 172.21.15.82:0/2599647309 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4974 (secure 0 0 0) 0x7f406009e2f0 con 0x7f406009bae0 2024-03-16T18:03:41.505 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:41.505 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5100,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3801030113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3801030113},{"type":"v1","addr":"172.21.15.82:6836","nonce":3801030113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2266371611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2266371611},{"type":"v1","addr":"172.21.15.82:6837","nonce":2266371611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/693422260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":693422260},{"type":"v1","addr":"172.21.15.90:6834","nonce":693422260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:58:49.876364+0000","modified":"2024-03-16T17:58:52.961483+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6835/2421794786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":2421794786},{"type":"v1","addr":"172.21.15.90:6835","nonce":2421794786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5118,"qdb_cluster":[5118]},"id":2}]} 2024-03-16T18:03:41.505 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 16 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 -- 172.21.15.82:0/2599647309 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f403c06b3b0 msgr2=0x7f403c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f403c06b3b0 0x7f403c06d870 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4050034000 tx=0x7f4050009030 comp rx=0 tx=0).stop 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 -- 172.21.15.82:0/2599647309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f406009bae0 msgr2=0x7f40600b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f406009bae0 0x7f40600b6d20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f404800e7e0 tx=0x7f404800eca0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 -- 172.21.15.82:0/2599647309 shutdown_connections 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f403c06b3b0 0x7f403c06d870 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40600ca860 0x7f406009ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f406009bae0 0x7f40600b6d20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 --2- 172.21.15.82:0/2599647309 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f4044003970 0x7f40600c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 -- 172.21.15.82:0/2599647309 >> 172.21.15.82:0/2599647309 conn(0x7f40600b37f0 msgr2=0x7f4060160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 -- 172.21.15.82:0/2599647309 shutdown_connections 2024-03-16T18:03:41.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.504+0000 7f4065000640 1 -- 172.21.15.82:0/2599647309 wait complete. 2024-03-16T18:03:41.526 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T18:03:41.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.731+0000 7faf3be00640 1 Processor -- start 2024-03-16T18:03:41.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.731+0000 7faf3be00640 1 -- start start 2024-03-16T18:03:41.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.732+0000 7faf3be00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf340cdf80 0x7faf340cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.732+0000 7faf3be00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf340cfe70 0x7faf34070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.732+0000 7faf3be00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf340b4b90 con 0x7faf341454d0 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.732+0000 7faf3be00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf340d0320 con 0x7faf340cfe70 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.732+0000 7faf3be00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf340cf500 con 0x7faf340cdf80 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.733+0000 7faf39e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf340cfe70 0x7faf34070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.733+0000 7faf39e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf340cfe70 0x7faf34070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35860/0 (socket says 172.21.15.82:35860) 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.733+0000 7faf39e00640 1 -- 172.21.15.82:0/723079972 learned_addr learned my addr 172.21.15.82:0/723079972 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.733+0000 7faf38a00640 1 --2- 172.21.15.82:0/723079972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf340cdf80 0x7faf340cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.733+0000 7faf39e00640 1 -- 172.21.15.82:0/723079972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf340cdf80 msgr2=0x7faf340cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.733+0000 7faf39e00640 1 --2- 172.21.15.82:0/723079972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf340cdf80 0x7faf340cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.733+0000 7faf39e00640 1 -- 172.21.15.82:0/723079972 >> v1:172.21.15.90:6789/0 conn(0x7faf341454d0 legacy=0x7faf340b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:41.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.733+0000 7faf39e00640 1 -- 172.21.15.82:0/723079972 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf341458b0 con 0x7faf340cfe70 2024-03-16T18:03:41.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.734+0000 7faf39e00640 1 --2- 172.21.15.82:0/723079972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cfe70 0x7faf34070980 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faf24009870 tx=0x7faf2402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94aea596597df9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:41.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.735+0000 7faf33e00640 1 -- 172.21.15.82:0/723079972 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24030840 con 0x7faf340cfe70 2024-03-16T18:03:41.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.735+0000 7faf33e00640 1 -- 172.21.15.82:0/723079972 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf240309a0 con 0x7faf340cfe70 2024-03-16T18:03:41.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.735+0000 7faf33e00640 1 -- 172.21.15.82:0/723079972 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24030d30 con 0x7faf340cfe70 2024-03-16T18:03:41.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.735+0000 7faf3be00640 1 -- 172.21.15.82:0/723079972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cfe70 msgr2=0x7faf34070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.735+0000 7faf3be00640 1 --2- 172.21.15.82:0/723079972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cfe70 0x7faf34070980 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faf24009870 tx=0x7faf2402b980 comp rx=0 tx=0).stop 2024-03-16T18:03:41.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.735+0000 7faf3be00640 1 -- 172.21.15.82:0/723079972 shutdown_connections 2024-03-16T18:03:41.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.735+0000 7faf3be00640 1 --2- 172.21.15.82:0/723079972 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf340cdf80 0x7faf340cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.735+0000 7faf3be00640 1 --2- 172.21.15.82:0/723079972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cfe70 0x7faf34070980 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.735+0000 7faf3be00640 1 -- 172.21.15.82:0/723079972 >> 172.21.15.82:0/723079972 conn(0x7faf3406ae70 msgr2=0x7faf3406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.736+0000 7faf3be00640 1 -- 172.21.15.82:0/723079972 shutdown_connections 2024-03-16T18:03:41.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.736+0000 7faf3be00640 1 -- 172.21.15.82:0/723079972 wait complete. 2024-03-16T18:03:41.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.737+0000 7faf3be00640 1 Processor -- start 2024-03-16T18:03:41.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.737+0000 7faf3be00640 1 -- start start 2024-03-16T18:03:41.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.738+0000 7faf3be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cdf80 0x7faf3415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.738+0000 7faf3be00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf340cfe70 0x7faf3406d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.738+0000 7faf3be00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf341454d0 0x7faf341f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.738+0000 7faf3be00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf341f3e50 con 0x7faf340cdf80 2024-03-16T18:03:41.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.738+0000 7faf3be00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3415f780 con 0x7faf341454d0 2024-03-16T18:03:41.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.738+0000 7faf3be00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3406ea30 con 0x7faf340cfe70 2024-03-16T18:03:41.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.738+0000 7faf39400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cdf80 0x7faf3415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.739+0000 7faf39400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cdf80 0x7faf3415f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35874/0 (socket says 172.21.15.82:35874) 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.739+0000 7faf39400640 1 -- 172.21.15.82:0/2543768664 learned_addr learned my addr 172.21.15.82:0/2543768664 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.739+0000 7faf39e00640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf341454d0 0x7faf341f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.739+0000 7faf38a00640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf340cfe70 0x7faf3406d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.739+0000 7faf39400640 1 -- 172.21.15.82:0/2543768664 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf340cfe70 msgr2=0x7faf3406d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.739+0000 7faf39400640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf340cfe70 0x7faf3406d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.739+0000 7faf39400640 1 -- 172.21.15.82:0/2543768664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf341454d0 msgr2=0x7faf341f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.739+0000 7faf39400640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf341454d0 0x7faf341f4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.739+0000 7faf39400640 1 -- 172.21.15.82:0/2543768664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240094d0 con 0x7faf340cdf80 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.739+0000 7faf39400640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cdf80 0x7faf3415f240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00c430 tx=0x7faf2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.740+0000 7faf31600640 1 -- 172.21.15.82:0/2543768664 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0052d0 con 0x7faf340cdf80 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.740+0000 7faf31600640 1 -- 172.21.15.82:0/2543768664 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c005430 con 0x7faf340cdf80 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.740+0000 7faf31600640 1 -- 172.21.15.82:0/2543768664 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c003830 con 0x7faf340cdf80 2024-03-16T18:03:41.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.740+0000 7faf3be00640 1 -- 172.21.15.82:0/2543768664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3406deb0 con 0x7faf340cdf80 2024-03-16T18:03:41.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.740+0000 7faf3be00640 1 -- 172.21.15.82:0/2543768664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3406e470 con 0x7faf340cdf80 2024-03-16T18:03:41.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.741+0000 7faf3be00640 1 -- 172.21.15.82:0/2543768664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf34067f30 con 0x7faf340cdf80 2024-03-16T18:03:41.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.742+0000 7faf31600640 1 -- 172.21.15.82:0/2543768664 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7faf2c003990 con 0x7faf340cdf80 2024-03-16T18:03:41.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.743+0000 7faf31600640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf1406b3b0 0x7faf1406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:41.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.743+0000 7faf31600640 1 -- 172.21.15.82:0/2543768664 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf2c089170 con 0x7faf340cdf80 2024-03-16T18:03:41.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.743+0000 7faf38a00640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf1406b3b0 0x7faf1406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:41.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.744+0000 7faf38a00640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf1406b3b0 0x7faf1406d870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faf340ccab0 tx=0x7faf2000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:41.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.748+0000 7faf31600640 1 -- 172.21.15.82:0/2543768664 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf2c053dd0 con 0x7faf340cdf80 2024-03-16T18:03:41.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.944+0000 7faf3be00640 1 -- 172.21.15.82:0/2543768664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf341437f0 con 0x7faf340cdf80 2024-03-16T18:03:41.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.946+0000 7faf31600640 1 -- 172.21.15.82:0/2543768664 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16633 (secure 0 0 0) 0x7faf2c057490 con 0x7faf340cdf80 2024-03-16T18:03:41.949 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:41.950 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:51.442896+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:58:47.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T17:58:48.893361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T18:03:41.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.948+0000 7faf3be00640 1 -- 172.21.15.82:0/2543768664 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf1406b3b0 msgr2=0x7faf1406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.948+0000 7faf3be00640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf1406b3b0 0x7faf1406d870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faf340ccab0 tx=0x7faf2000b040 comp rx=0 tx=0).stop 2024-03-16T18:03:41.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.948+0000 7faf3be00640 1 -- 172.21.15.82:0/2543768664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cdf80 msgr2=0x7faf3415f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:41.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.948+0000 7faf3be00640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cdf80 0x7faf3415f240 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00c430 tx=0x7faf2c00c8f0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.948+0000 7faf3be00640 1 -- 172.21.15.82:0/2543768664 shutdown_connections 2024-03-16T18:03:41.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.948+0000 7faf3be00640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faf1406b3b0 0x7faf1406d870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.948+0000 7faf3be00640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf341454d0 0x7faf341f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.949+0000 7faf3be00640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faf340cfe70 0x7faf3406d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.949+0000 7faf3be00640 1 --2- 172.21.15.82:0/2543768664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf340cdf80 0x7faf3415f240 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:41.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.949+0000 7faf3be00640 1 -- 172.21.15.82:0/2543768664 >> 172.21.15.82:0/2543768664 conn(0x7faf3406ae70 msgr2=0x7faf34160230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:41.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.949+0000 7faf3be00640 1 -- 172.21.15.82:0/2543768664 shutdown_connections 2024-03-16T18:03:41.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:41.949+0000 7faf3be00640 1 -- 172.21.15.82:0/2543768664 wait complete. 2024-03-16T18:03:41.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T17:57:34.080848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T17:58:47.770770+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T17:58:48.893361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T18:03:41.973 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T18:03:42.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.186+0000 7f99d2600640 1 Processor -- start 2024-03-16T18:03:42.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.186+0000 7f99d2600640 1 -- start start 2024-03-16T18:03:42.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.187+0000 7f99d2600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f99cc0d2ab0 0x7f99cc15e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.187+0000 7f99d2600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d48e0 0x7f99cc0ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.187+0000 7f99d2600640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99cc0b33c0 con 0x7f99cc15e040 2024-03-16T18:03:42.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.187+0000 7f99d2600640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc0d1cd0 con 0x7f99cc0d2ab0 2024-03-16T18:03:42.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.187+0000 7f99d2600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0d1fb0 con 0x7f99cc0d48e0 2024-03-16T18:03:42.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.187+0000 7f99cb400640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f99cc15e040 0x7f99cc0d23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.82:39686/0 (socket says 172.21.15.82:39686) 2024-03-16T18:03:42.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.187+0000 7f99cb400640 1 -- 172.21.15.82:0/3856312969 learned_addr learned my addr 172.21.15.82:0/3856312969 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:42.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.187+0000 7f99cbe00640 1 --2- 172.21.15.82:0/3856312969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d48e0 0x7f99cc0ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.187+0000 7f99c2a00640 1 --2- 172.21.15.82:0/3856312969 >> v2:172.21.15.90:3300/0 conn(0x7f99cc0d2ab0 0x7f99cc15e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.188+0000 7f99caa00640 1 -- 172.21.15.82:0/3856312969 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190792440 0 0) 0x7f99cc0b33c0 con 0x7f99cc15e040 2024-03-16T18:03:42.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.188+0000 7f99caa00640 1 -- 172.21.15.82:0/3856312969 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99b0003520 con 0x7f99cc15e040 2024-03-16T18:03:42.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.188+0000 7f99cbe00640 1 -- 172.21.15.82:0/3856312969 >> v2:172.21.15.90:3300/0 conn(0x7f99cc0d2ab0 msgr2=0x7f99cc15e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:42.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.188+0000 7f99cbe00640 1 --2- 172.21.15.82:0/3856312969 >> v2:172.21.15.90:3300/0 conn(0x7f99cc0d2ab0 0x7f99cc15e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.188+0000 7f99cbe00640 1 -- 172.21.15.82:0/3856312969 >> v1:172.21.15.90:6789/0 conn(0x7f99cc15e040 legacy=0x7f99cc0d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.188+0000 7f99cbe00640 1 -- 172.21.15.82:0/3856312969 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc0b3610 con 0x7f99cc0d48e0 2024-03-16T18:03:42.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.189+0000 7f99cbe00640 1 --2- 172.21.15.82:0/3856312969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d48e0 0x7f99cc0ce9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99c40097a0 tx=0x7f99c402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe6c9b031ebaed6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:42.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99caa00640 1 -- 172.21.15.82:0/3856312969 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c402e070 con 0x7f99cc0d48e0 2024-03-16T18:03:42.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99caa00640 1 -- 172.21.15.82:0/3856312969 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c40315a0 con 0x7f99cc0d48e0 2024-03-16T18:03:42.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99caa00640 1 -- 172.21.15.82:0/3856312969 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4031910 con 0x7f99cc0d48e0 2024-03-16T18:03:42.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99d2600640 1 -- 172.21.15.82:0/3856312969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d48e0 msgr2=0x7f99cc0ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99d2600640 1 --2- 172.21.15.82:0/3856312969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d48e0 0x7f99cc0ce9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99c40097a0 tx=0x7f99c402b840 comp rx=0 tx=0).stop 2024-03-16T18:03:42.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99d2600640 1 -- 172.21.15.82:0/3856312969 shutdown_connections 2024-03-16T18:03:42.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99d2600640 1 --2- 172.21.15.82:0/3856312969 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d48e0 0x7f99cc0ce9d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99d2600640 1 --2- 172.21.15.82:0/3856312969 >> v2:172.21.15.90:3300/0 conn(0x7f99cc0d2ab0 0x7f99cc15e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99d2600640 1 -- 172.21.15.82:0/3856312969 >> 172.21.15.82:0/3856312969 conn(0x7f99cc0d3dc0 msgr2=0x7f99cc0d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:42.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99d2600640 1 -- 172.21.15.82:0/3856312969 shutdown_connections 2024-03-16T18:03:42.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.190+0000 7f99d2600640 1 -- 172.21.15.82:0/3856312969 wait complete. 2024-03-16T18:03:42.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.192+0000 7f99d2600640 1 Processor -- start 2024-03-16T18:03:42.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.192+0000 7f99d2600640 1 -- start start 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.193+0000 7f99d2600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d2ab0 0x7f99cc0b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.193+0000 7f99d2600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99cc0d48e0 0x7f99cc153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.193+0000 7f99d2600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99cc15e040 0x7f99cc0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.193+0000 7f99d2600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc15b6a0 con 0x7f99cc0d48e0 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.193+0000 7f99d2600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0b7ad0 con 0x7f99cc15e040 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.193+0000 7f99d2600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc153660 con 0x7f99cc0d2ab0 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99c2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99cc0d48e0 0x7f99cc153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99c2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99cc0d48e0 0x7f99cc153120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35876/0 (socket says 172.21.15.82:35876) 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99c2a00640 1 -- 172.21.15.82:0/2642286547 learned_addr learned my addr 172.21.15.82:0/2642286547 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99cb400640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d2ab0 0x7f99cc0b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99cbe00640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99cc15e040 0x7f99cc0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99c2a00640 1 -- 172.21.15.82:0/2642286547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d2ab0 msgr2=0x7f99cc0b7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99c2a00640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d2ab0 0x7f99cc0b7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99c2a00640 1 -- 172.21.15.82:0/2642286547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99cc15e040 msgr2=0x7f99cc0ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99c2a00640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99cc15e040 0x7f99cc0ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99c2a00640 1 -- 172.21.15.82:0/2642286547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4009400 con 0x7f99cc0d48e0 2024-03-16T18:03:42.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99cbe00640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99cc15e040 0x7f99cc0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:42.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.194+0000 7f99c2a00640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99cc0d48e0 0x7f99cc153120 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f99b400e9e0 tx=0x7f99b400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:42.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.195+0000 7f99c3e00640 1 -- 172.21.15.82:0/2642286547 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4009670 con 0x7f99cc0d48e0 2024-03-16T18:03:42.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.195+0000 7f99c3e00640 1 -- 172.21.15.82:0/2642286547 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b400c9d0 con 0x7f99cc0d48e0 2024-03-16T18:03:42.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.195+0000 7f99d2600640 1 -- 172.21.15.82:0/2642286547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc0b6ec0 con 0x7f99cc0d48e0 2024-03-16T18:03:42.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.195+0000 7f99c3e00640 1 -- 172.21.15.82:0/2642286547 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b400cd80 con 0x7f99cc0d48e0 2024-03-16T18:03:42.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.195+0000 7f99d2600640 1 -- 172.21.15.82:0/2642286547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc0b7480 con 0x7f99cc0d48e0 2024-03-16T18:03:42.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.196+0000 7f99d2600640 1 -- 172.21.15.82:0/2642286547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc067f30 con 0x7f99cc0d48e0 2024-03-16T18:03:42.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.197+0000 7f99c3e00640 1 -- 172.21.15.82:0/2642286547 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f99b4013470 con 0x7f99cc0d48e0 2024-03-16T18:03:42.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.198+0000 7f99c3e00640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99a806b3b0 0x7f99a806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.198+0000 7f99c3e00640 1 -- 172.21.15.82:0/2642286547 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99b408a2c0 con 0x7f99cc0d48e0 2024-03-16T18:03:42.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.198+0000 7f99cb400640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99a806b3b0 0x7f99a806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.199+0000 7f99cb400640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99a806b3b0 0x7f99a806d870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f99bc001560 tx=0x7f99bc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:42.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.203+0000 7f99c3e00640 1 -- 172.21.15.82:0/2642286547 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f99b4054df0 con 0x7f99cc0d48e0 2024-03-16T18:03:42.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.400+0000 7f99d2600640 1 -- 172.21.15.82:0/2642286547 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99cc1e5350 con 0x7f99cc0d48e0 2024-03-16T18:03:42.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.401+0000 7f99c3e00640 1 -- 172.21.15.82:0/2642286547 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99b40584b0 con 0x7f99cc0d48e0 2024-03-16T18:03:42.405 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 -- 172.21.15.82:0/2642286547 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99a806b3b0 msgr2=0x7f99a806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99a806b3b0 0x7f99a806d870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f99bc001560 tx=0x7f99bc009040 comp rx=0 tx=0).stop 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 -- 172.21.15.82:0/2642286547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99cc0d48e0 msgr2=0x7f99cc153120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99cc0d48e0 0x7f99cc153120 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f99b400e9e0 tx=0x7f99b400eea0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 -- 172.21.15.82:0/2642286547 shutdown_connections 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f99a806b3b0 0x7f99a806d870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99cc15e040 0x7f99cc0ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99cc0d48e0 0x7f99cc153120 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 --2- 172.21.15.82:0/2642286547 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f99cc0d2ab0 0x7f99cc0b7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 -- 172.21.15.82:0/2642286547 >> 172.21.15.82:0/2642286547 conn(0x7f99cc0d3dc0 msgr2=0x7f99cc06c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 -- 172.21.15.82:0/2642286547 shutdown_connections 2024-03-16T18:03:42.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.405+0000 7f99d2600640 1 -- 172.21.15.82:0/2642286547 wait complete. 2024-03-16T18:03:42.428 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T18:03:42.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.637+0000 7f9e18800640 1 Processor -- start 2024-03-16T18:03:42.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.637+0000 7f9e18800640 1 -- start start 2024-03-16T18:03:42.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.639+0000 7f9e18800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e1006e940 0x7f9e100cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.639+0000 7f9e18800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9e100cc850 0x7f9e100cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.639+0000 7f9e18800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1015c370 con 0x7f9e10159960 2024-03-16T18:03:42.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.639+0000 7f9e18800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e10159d40 con 0x7f9e100cc850 2024-03-16T18:03:42.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.639+0000 7f9e18800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1015cff0 con 0x7f9e1006e940 2024-03-16T18:03:42.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.639+0000 7f9e15400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e1006e940 0x7f9e100cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.639+0000 7f9e16800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9e100cc850 0x7f9e100cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.639+0000 7f9e15e00640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7f9e10159960 0x7f9e1006e230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.82:39696/0 (socket says 172.21.15.82:39696) 2024-03-16T18:03:42.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.639+0000 7f9e15e00640 1 -- 172.21.15.82:0/3301086838 learned_addr learned my addr 172.21.15.82:0/3301086838 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:42.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.640+0000 7f9e16800640 1 -- 172.21.15.82:0/3301086838 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e1006e940 msgr2=0x7f9e100cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:42.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.640+0000 7f9e16800640 1 --2- 172.21.15.82:0/3301086838 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e1006e940 0x7f9e100cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.640+0000 7f9e16800640 1 -- 172.21.15.82:0/3301086838 >> v1:172.21.15.90:6789/0 conn(0x7f9e10159960 legacy=0x7f9e1006e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.640+0000 7f9e16800640 1 -- 172.21.15.82:0/3301086838 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1015c650 con 0x7f9e100cc850 2024-03-16T18:03:42.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.640+0000 7f9e15400640 1 --2- 172.21.15.82:0/3301086838 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e1006e940 0x7f9e100cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T18:03:42.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.641+0000 7f9e16800640 1 --2- 172.21.15.82:0/3301086838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e100cc850 0x7f9e100cd540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009d90 tx=0x7f9e0c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee64bd94f625d2bd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:42.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.642+0000 7f9e14a00640 1 -- 172.21.15.82:0/3301086838 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c02cbc0 con 0x7f9e100cc850 2024-03-16T18:03:42.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.642+0000 7f9e14a00640 1 -- 172.21.15.82:0/3301086838 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0c02cd20 con 0x7f9e100cc850 2024-03-16T18:03:42.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.642+0000 7f9e14a00640 1 -- 172.21.15.82:0/3301086838 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c034660 con 0x7f9e100cc850 2024-03-16T18:03:42.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.642+0000 7f9e18800640 1 -- 172.21.15.82:0/3301086838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e100cc850 msgr2=0x7f9e100cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.642+0000 7f9e18800640 1 --2- 172.21.15.82:0/3301086838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e100cc850 0x7f9e100cd540 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009d90 tx=0x7f9e0c02be10 comp rx=0 tx=0).stop 2024-03-16T18:03:42.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.642+0000 7f9e18800640 1 -- 172.21.15.82:0/3301086838 shutdown_connections 2024-03-16T18:03:42.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.642+0000 7f9e18800640 1 --2- 172.21.15.82:0/3301086838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e100cc850 0x7f9e100cd540 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.642+0000 7f9e18800640 1 --2- 172.21.15.82:0/3301086838 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e1006e940 0x7f9e100cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.642+0000 7f9e18800640 1 -- 172.21.15.82:0/3301086838 >> 172.21.15.82:0/3301086838 conn(0x7f9e100b1640 msgr2=0x7f9e100b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:42.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.643+0000 7f9e18800640 1 -- 172.21.15.82:0/3301086838 shutdown_connections 2024-03-16T18:03:42.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.643+0000 7f9e18800640 1 -- 172.21.15.82:0/3301086838 wait complete. 2024-03-16T18:03:42.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.644+0000 7f9e18800640 1 Processor -- start 2024-03-16T18:03:42.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.644+0000 7f9e18800640 1 -- start start 2024-03-16T18:03:42.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.645+0000 7f9e18800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e1006e940 0x7f9e10137f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.645+0000 7f9e18800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e100cc850 0x7f9e10158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.645+0000 7f9e18800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e10159960 0x7f9e100c9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.645+0000 7f9e18800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e101384a0 con 0x7f9e100cc850 2024-03-16T18:03:42.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.645+0000 7f9e18800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e10158dc0 con 0x7f9e1006e940 2024-03-16T18:03:42.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.645+0000 7f9e18800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e100ca350 con 0x7f9e10159960 2024-03-16T18:03:42.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e15400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e100cc850 0x7f9e10158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e15400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e100cc850 0x7f9e10158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35878/0 (socket says 172.21.15.82:35878) 2024-03-16T18:03:42.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e15400640 1 -- 172.21.15.82:0/1189614651 learned_addr learned my addr 172.21.15.82:0/1189614651 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:42.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e15e00640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e1006e940 0x7f9e10137f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e16800640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e10159960 0x7f9e100c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e15400640 1 -- 172.21.15.82:0/1189614651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e10159960 msgr2=0x7f9e100c9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e15400640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e10159960 0x7f9e100c9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e15400640 1 -- 172.21.15.82:0/1189614651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e1006e940 msgr2=0x7f9e10137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e15400640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e1006e940 0x7f9e10137f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e15400640 1 -- 172.21.15.82:0/1189614651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e00009400 con 0x7f9e100cc850 2024-03-16T18:03:42.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.646+0000 7f9e15400640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e100cc850 0x7f9e10158880 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000ebe0 tx=0x7f9e0000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:42.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.647+0000 7f9e0a000640 1 -- 172.21.15.82:0/1189614651 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00009760 con 0x7f9e100cc850 2024-03-16T18:03:42.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.647+0000 7f9e0a000640 1 -- 172.21.15.82:0/1189614651 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0000cc30 con 0x7f9e100cc850 2024-03-16T18:03:42.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.647+0000 7f9e0a000640 1 -- 172.21.15.82:0/1189614651 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0000eee0 con 0x7f9e100cc850 2024-03-16T18:03:42.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.647+0000 7f9e18800640 1 -- 172.21.15.82:0/1189614651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0c009a70 con 0x7f9e100cc850 2024-03-16T18:03:42.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.647+0000 7f9e18800640 1 -- 172.21.15.82:0/1189614651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e100ca910 con 0x7f9e100cc850 2024-03-16T18:03:42.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.648+0000 7f9e18800640 1 -- 172.21.15.82:0/1189614651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1006c980 con 0x7f9e100cc850 2024-03-16T18:03:42.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.649+0000 7f9e0a000640 1 -- 172.21.15.82:0/1189614651 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f9e00013470 con 0x7f9e100cc850 2024-03-16T18:03:42.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.650+0000 7f9e0a000640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9df406b3b0 0x7f9df406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:42.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.650+0000 7f9e0a000640 1 -- 172.21.15.82:0/1189614651 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e00089f00 con 0x7f9e100cc850 2024-03-16T18:03:42.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.650+0000 7f9e15e00640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9df406b3b0 0x7f9df406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:42.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.651+0000 7f9e15e00640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9df406b3b0 0x7f9df406d870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e04000f60 tx=0x7f9e04009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:42.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.655+0000 7f9e0a000640 1 -- 172.21.15.82:0/1189614651 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e1006c980 con 0x7f9e100cc850 2024-03-16T18:03:42.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.850+0000 7f9e18800640 1 -- 172.21.15.82:0/1189614651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e1006c980 con 0x7f9e100cc850 2024-03-16T18:03:42.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.850+0000 7f9e0a000640 1 -- 172.21.15.82:0/1189614651 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e1006c980 con 0x7f9e100cc850 2024-03-16T18:03:42.854 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-03-16T18:03:42.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 -- 172.21.15.82:0/1189614651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9df406b3b0 msgr2=0x7f9df406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9df406b3b0 0x7f9df406d870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e04000f60 tx=0x7f9e04009040 comp rx=0 tx=0).stop 2024-03-16T18:03:42.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 -- 172.21.15.82:0/1189614651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e100cc850 msgr2=0x7f9e10158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:42.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e100cc850 0x7f9e10158880 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000ebe0 tx=0x7f9e0000c520 comp rx=0 tx=0).stop 2024-03-16T18:03:42.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 -- 172.21.15.82:0/1189614651 shutdown_connections 2024-03-16T18:03:42.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9df406b3b0 0x7f9df406d870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9e10159960 0x7f9e100c9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e100cc850 0x7f9e10158880 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 --2- 172.21.15.82:0/1189614651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e1006e940 0x7f9e10137f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:42.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 -- 172.21.15.82:0/1189614651 >> 172.21.15.82:0/1189614651 conn(0x7f9e100b1640 msgr2=0x7f9e10071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:42.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 -- 172.21.15.82:0/1189614651 shutdown_connections 2024-03-16T18:03:42.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:42.853+0000 7f9e18800640 1 -- 172.21.15.82:0/1189614651 wait complete. 2024-03-16T18:03:42.875 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T18:03:43.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.086+0000 7fd86fc00640 1 Processor -- start 2024-03-16T18:03:43.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.086+0000 7fd86fc00640 1 -- start start 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.087+0000 7fd86fc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd86806e940 0x7fd8680cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.087+0000 7fd86fc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd8680cc850 0x7fd8680cd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd86fc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86815c370 con 0x7fd868159960 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd86fc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd868159d40 con 0x7fd8680cc850 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd86fc00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86815cff0 con 0x7fd86806e940 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd86dc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd8680cc850 0x7fd8680cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd86d200640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd868159960 0x7fd86806e230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51888/0 (socket says 172.21.15.82:51888) 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd86d200640 1 -- 172.21.15.82:0/1204285544 learned_addr learned my addr 172.21.15.82:0/1204285544 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd867e00640 1 --2- 172.21.15.82:0/1204285544 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd86806e940 0x7fd8680cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd86dc00640 1 -- 172.21.15.82:0/1204285544 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd86806e940 msgr2=0x7fd8680cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:43.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd86dc00640 1 --2- 172.21.15.82:0/1204285544 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd86806e940 0x7fd8680cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd86dc00640 1 -- 172.21.15.82:0/1204285544 >> v1:172.21.15.82:6789/0 conn(0x7fd868159960 legacy=0x7fd86806e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd867400640 1 -- 172.21.15.82:0/1204285544 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369176893 0 0) 0x7fd86815c370 con 0x7fd868159960 2024-03-16T18:03:43.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.088+0000 7fd86dc00640 1 -- 172.21.15.82:0/1204285544 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86815c650 con 0x7fd8680cc850 2024-03-16T18:03:43.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.090+0000 7fd86dc00640 1 --2- 172.21.15.82:0/1204285544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8680cc850 0x7fd8680cd540 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd860009d90 tx=0x7fd86002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f1cbda9b391c4b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:43.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.090+0000 7fd867400640 1 -- 172.21.15.82:0/1204285544 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86002cbc0 con 0x7fd8680cc850 2024-03-16T18:03:43.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.090+0000 7fd867400640 1 -- 172.21.15.82:0/1204285544 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86002cd20 con 0x7fd8680cc850 2024-03-16T18:03:43.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.090+0000 7fd867400640 1 -- 172.21.15.82:0/1204285544 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860034660 con 0x7fd8680cc850 2024-03-16T18:03:43.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.090+0000 7fd86fc00640 1 -- 172.21.15.82:0/1204285544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8680cc850 msgr2=0x7fd8680cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.090+0000 7fd86fc00640 1 --2- 172.21.15.82:0/1204285544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8680cc850 0x7fd8680cd540 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd860009d90 tx=0x7fd86002be10 comp rx=0 tx=0).stop 2024-03-16T18:03:43.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.091+0000 7fd86fc00640 1 -- 172.21.15.82:0/1204285544 shutdown_connections 2024-03-16T18:03:43.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.091+0000 7fd86fc00640 1 --2- 172.21.15.82:0/1204285544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8680cc850 0x7fd8680cd540 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.091+0000 7fd86fc00640 1 --2- 172.21.15.82:0/1204285544 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd86806e940 0x7fd8680cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.091+0000 7fd86fc00640 1 -- 172.21.15.82:0/1204285544 >> 172.21.15.82:0/1204285544 conn(0x7fd8680b1640 msgr2=0x7fd8680b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:43.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.091+0000 7fd86fc00640 1 -- 172.21.15.82:0/1204285544 shutdown_connections 2024-03-16T18:03:43.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.091+0000 7fd86fc00640 1 -- 172.21.15.82:0/1204285544 wait complete. 2024-03-16T18:03:43.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.092+0000 7fd86fc00640 1 Processor -- start 2024-03-16T18:03:43.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.093+0000 7fd86fc00640 1 -- start start 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.093+0000 7fd86fc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86806e940 0x7fd868142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.093+0000 7fd86fc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8680cc850 0x7fd86813fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.093+0000 7fd86fc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd868159960 0x7fd868140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86fc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8681415e0 con 0x7fd86806e940 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86fc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8681418c0 con 0x7fd868159960 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86fc00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd868141ba0 con 0x7fd8680cc850 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86806e940 0x7fd868142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86806e940 0x7fd868142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35900/0 (socket says 172.21.15.82:35900) 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86d200640 1 -- 172.21.15.82:0/3577861680 learned_addr learned my addr 172.21.15.82:0/3577861680 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd867e00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8680cc850 0x7fd86813fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86dc00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd868159960 0x7fd868140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86d200640 1 -- 172.21.15.82:0/3577861680 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8680cc850 msgr2=0x7fd86813fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86d200640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8680cc850 0x7fd86813fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86d200640 1 -- 172.21.15.82:0/3577861680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd868159960 msgr2=0x7fd868140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86d200640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd868159960 0x7fd868140c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.094+0000 7fd86d200640 1 -- 172.21.15.82:0/3577861680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd860009a70 con 0x7fd86806e940 2024-03-16T18:03:43.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.095+0000 7fd86d200640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86806e940 0x7fd868142680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c430 tx=0x7fd85c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:43.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.095+0000 7fd864c00640 1 -- 172.21.15.82:0/3577861680 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0074e0 con 0x7fd86806e940 2024-03-16T18:03:43.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.095+0000 7fd864c00640 1 -- 172.21.15.82:0/3577861680 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c007660 con 0x7fd86806e940 2024-03-16T18:03:43.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.095+0000 7fd864c00640 1 -- 172.21.15.82:0/3577861680 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c004a20 con 0x7fd86806e940 2024-03-16T18:03:43.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.095+0000 7fd86fc00640 1 -- 172.21.15.82:0/3577861680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd868141e80 con 0x7fd86806e940 2024-03-16T18:03:43.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.095+0000 7fd86fc00640 1 -- 172.21.15.82:0/3577861680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd868140140 con 0x7fd86806e940 2024-03-16T18:03:43.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.096+0000 7fd86fc00640 1 -- 172.21.15.82:0/3577861680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86806c980 con 0x7fd86806e940 2024-03-16T18:03:43.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.097+0000 7fd864c00640 1 -- 172.21.15.82:0/3577861680 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fd85c004b80 con 0x7fd86806e940 2024-03-16T18:03:43.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.098+0000 7fd864c00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd84806b3b0 0x7fd84806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.098+0000 7fd864c00640 1 -- 172.21.15.82:0/3577861680 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd85c0899f0 con 0x7fd86806e940 2024-03-16T18:03:43.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.098+0000 7fd867e00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd84806b3b0 0x7fd84806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.099+0000 7fd867e00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd84806b3b0 0x7fd84806d870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd868159210 tx=0x7fd854008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:43.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.102+0000 7fd864c00640 1 -- 172.21.15.82:0/3577861680 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd86806c980 con 0x7fd86806e940 2024-03-16T18:03:43.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.302+0000 7fd86fc00640 1 -- 172.21.15.82:0/3577861680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd86806c980 con 0x7fd86806e940 2024-03-16T18:03:43.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.303+0000 7fd864c00640 1 -- 172.21.15.82:0/3577861680 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd86806c980 con 0x7fd86806e940 2024-03-16T18:03:43.305 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-03-16T18:03:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 -- 172.21.15.82:0/3577861680 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd84806b3b0 msgr2=0x7fd84806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd84806b3b0 0x7fd84806d870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd868159210 tx=0x7fd854008040 comp rx=0 tx=0).stop 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 -- 172.21.15.82:0/3577861680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86806e940 msgr2=0x7fd868142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86806e940 0x7fd868142680 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c430 tx=0x7fd85c00c8f0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 -- 172.21.15.82:0/3577861680 shutdown_connections 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd84806b3b0 0x7fd84806d870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd868159960 0x7fd868140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8680cc850 0x7fd86813fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 --2- 172.21.15.82:0/3577861680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86806e940 0x7fd868142680 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 -- 172.21.15.82:0/3577861680 >> 172.21.15.82:0/3577861680 conn(0x7fd8680b1640 msgr2=0x7fd8680c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 -- 172.21.15.82:0/3577861680 shutdown_connections 2024-03-16T18:03:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.306+0000 7fd86fc00640 1 -- 172.21.15.82:0/3577861680 wait complete. 2024-03-16T18:03:43.327 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T18:03:43.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.524+0000 7fdc24a00640 1 Processor -- start 2024-03-16T18:03:43.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.525+0000 7fdc24a00640 1 -- start start 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.526+0000 7fdc24a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc200aee40 0x7fdc2006d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.526+0000 7fdc24a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc20159990 0x7fdc2006d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.526+0000 7fdc24a00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2006dab0 con 0x7fdc201595b0 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.526+0000 7fdc24a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2015d1c0 con 0x7fdc200aee40 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.526+0000 7fdc24a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200b3370 con 0x7fdc20159990 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.527+0000 7fdc15e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc200aee40 0x7fdc2006d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.527+0000 7fdc15e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc200aee40 0x7fdc2006d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35908/0 (socket says 172.21.15.82:35908) 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.527+0000 7fdc15e00640 1 -- 172.21.15.82:0/3129360360 learned_addr learned my addr 172.21.15.82:0/3129360360 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.527+0000 7fdc1e800640 1 --1- 172.21.15.82:0/3129360360 >> v1:172.21.15.82:6789/0 conn(0x7fdc201595b0 0x7fdc200ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51892/0 (socket says 172.21.15.82:51892) 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.527+0000 7fdc1f200640 1 --2- 172.21.15.82:0/3129360360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc20159990 0x7fdc2006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.527+0000 7fdc15e00640 1 -- 172.21.15.82:0/3129360360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc20159990 msgr2=0x7fdc2006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.527+0000 7fdc15e00640 1 --2- 172.21.15.82:0/3129360360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc20159990 0x7fdc2006d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.527+0000 7fdc15e00640 1 -- 172.21.15.82:0/3129360360 >> v1:172.21.15.82:6789/0 conn(0x7fdc201595b0 legacy=0x7fdc200ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.527+0000 7fdc15e00640 1 -- 172.21.15.82:0/3129360360 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2006dd20 con 0x7fdc200aee40 2024-03-16T18:03:43.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.527+0000 7fdc1de00640 1 -- 172.21.15.82:0/3129360360 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179056165 0 0) 0x7fdc2006dab0 con 0x7fdc201595b0 2024-03-16T18:03:43.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.528+0000 7fdc15e00640 1 --2- 172.21.15.82:0/3129360360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc200aee40 0x7fdc2006d030 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009870 tx=0x7fdc0802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a729486571fd967f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:43.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc1de00640 1 -- 172.21.15.82:0/3129360360 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08030840 con 0x7fdc200aee40 2024-03-16T18:03:43.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc1de00640 1 -- 172.21.15.82:0/3129360360 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc080309a0 con 0x7fdc200aee40 2024-03-16T18:03:43.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc1de00640 1 -- 172.21.15.82:0/3129360360 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08030d30 con 0x7fdc200aee40 2024-03-16T18:03:43.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc24a00640 1 -- 172.21.15.82:0/3129360360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc200aee40 msgr2=0x7fdc2006d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc24a00640 1 --2- 172.21.15.82:0/3129360360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc200aee40 0x7fdc2006d030 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009870 tx=0x7fdc0802b980 comp rx=0 tx=0).stop 2024-03-16T18:03:43.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc24a00640 1 -- 172.21.15.82:0/3129360360 shutdown_connections 2024-03-16T18:03:43.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc24a00640 1 --2- 172.21.15.82:0/3129360360 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc20159990 0x7fdc2006d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc24a00640 1 --2- 172.21.15.82:0/3129360360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc200aee40 0x7fdc2006d030 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc24a00640 1 -- 172.21.15.82:0/3129360360 >> 172.21.15.82:0/3129360360 conn(0x7fdc2006ae70 msgr2=0x7fdc2006b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:43.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc24a00640 1 -- 172.21.15.82:0/3129360360 shutdown_connections 2024-03-16T18:03:43.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.529+0000 7fdc24a00640 1 -- 172.21.15.82:0/3129360360 wait complete. 2024-03-16T18:03:43.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.531+0000 7fdc24a00640 1 Processor -- start 2024-03-16T18:03:43.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.531+0000 7fdc24a00640 1 -- start start 2024-03-16T18:03:43.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.532+0000 7fdc24a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc200aee40 0x7fdc20151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.532+0000 7fdc24a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc201595b0 0x7fdc201f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.532+0000 7fdc24a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc20159990 0x7fdc2014a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.532+0000 7fdc24a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc201530e0 con 0x7fdc201595b0 2024-03-16T18:03:43.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.532+0000 7fdc24a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc201f8430 con 0x7fdc200aee40 2024-03-16T18:03:43.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.532+0000 7fdc24a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20153570 con 0x7fdc20159990 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.532+0000 7fdc15e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc201595b0 0x7fdc201f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.533+0000 7fdc15e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc201595b0 0x7fdc201f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35922/0 (socket says 172.21.15.82:35922) 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.533+0000 7fdc15e00640 1 -- 172.21.15.82:0/1702799745 learned_addr learned my addr 172.21.15.82:0/1702799745 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.533+0000 7fdc1e800640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc200aee40 0x7fdc20151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.533+0000 7fdc15e00640 1 -- 172.21.15.82:0/1702799745 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc20159990 msgr2=0x7fdc2014a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.533+0000 7fdc15e00640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc20159990 0x7fdc2014a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.533+0000 7fdc15e00640 1 -- 172.21.15.82:0/1702799745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc200aee40 msgr2=0x7fdc20151e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.533+0000 7fdc15e00640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc200aee40 0x7fdc20151e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.533+0000 7fdc15e00640 1 -- 172.21.15.82:0/1702799745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc080094d0 con 0x7fdc201595b0 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.533+0000 7fdc1e800640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc200aee40 0x7fdc20151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.533+0000 7fdc15e00640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc201595b0 0x7fdc201f7ef0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009870 tx=0x7fdc08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.534+0000 7fdc17400640 1 -- 172.21.15.82:0/1702799745 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08039c90 con 0x7fdc201595b0 2024-03-16T18:03:43.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.534+0000 7fdc17400640 1 -- 172.21.15.82:0/1702799745 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc08039df0 con 0x7fdc201595b0 2024-03-16T18:03:43.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.534+0000 7fdc17400640 1 -- 172.21.15.82:0/1702799745 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08043620 con 0x7fdc201595b0 2024-03-16T18:03:43.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.534+0000 7fdc24a00640 1 -- 172.21.15.82:0/1702799745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc201524d0 con 0x7fdc201595b0 2024-03-16T18:03:43.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.534+0000 7fdc24a00640 1 -- 172.21.15.82:0/1702799745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc20152a90 con 0x7fdc201595b0 2024-03-16T18:03:43.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.535+0000 7fdc24a00640 1 -- 172.21.15.82:0/1702799745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc200b2430 con 0x7fdc201595b0 2024-03-16T18:03:43.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.536+0000 7fdc17400640 1 -- 172.21.15.82:0/1702799745 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fdc08043780 con 0x7fdc201595b0 2024-03-16T18:03:43.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.537+0000 7fdc17400640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdbfc06b3b0 0x7fdbfc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.537+0000 7fdc17400640 1 -- 172.21.15.82:0/1702799745 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc0803e070 con 0x7fdc201595b0 2024-03-16T18:03:43.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.537+0000 7fdc1e800640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdbfc06b3b0 0x7fdbfc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.538+0000 7fdc1e800640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdbfc06b3b0 0x7fdbfc06d870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdc10000f60 tx=0x7fdc10009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:43.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.541+0000 7fdc17400640 1 -- 172.21.15.82:0/1702799745 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdc080779b0 con 0x7fdc201595b0 2024-03-16T18:03:43.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.773+0000 7fdc24a00640 1 -- 172.21.15.82:0/1702799745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc200b2f50 con 0x7fdc201595b0 2024-03-16T18:03:43.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.774+0000 7fdc17400640 1 -- 172.21.15.82:0/1702799745 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4974 (secure 0 0 0) 0x7fdc08034090 con 0x7fdc201595b0 2024-03-16T18:03:43.777 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:43.778 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5100,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3801030113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3801030113},{"type":"v1","addr":"172.21.15.82:6836","nonce":3801030113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2266371611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2266371611},{"type":"v1","addr":"172.21.15.82:6837","nonce":2266371611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/693422260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":693422260},{"type":"v1","addr":"172.21.15.90:6834","nonce":693422260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:58:49.876364+0000","modified":"2024-03-16T17:58:52.961483+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6835/2421794786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":2421794786},{"type":"v1","addr":"172.21.15.90:6835","nonce":2421794786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5118,"qdb_cluster":[5118]},"id":2}]} 2024-03-16T18:03:43.778 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 16 2024-03-16T18:03:43.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.776+0000 7fdc24a00640 1 -- 172.21.15.82:0/1702799745 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdbfc06b3b0 msgr2=0x7fdbfc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.776+0000 7fdc24a00640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdbfc06b3b0 0x7fdbfc06d870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdc10000f60 tx=0x7fdc10009040 comp rx=0 tx=0).stop 2024-03-16T18:03:43.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.777+0000 7fdc24a00640 1 -- 172.21.15.82:0/1702799745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc201595b0 msgr2=0x7fdc201f7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.777+0000 7fdc24a00640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc201595b0 0x7fdc201f7ef0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdc08009870 tx=0x7fdc08004730 comp rx=0 tx=0).stop 2024-03-16T18:03:43.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.777+0000 7fdc24a00640 1 -- 172.21.15.82:0/1702799745 shutdown_connections 2024-03-16T18:03:43.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.777+0000 7fdc24a00640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fdbfc06b3b0 0x7fdbfc06d870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.777+0000 7fdc24a00640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdc20159990 0x7fdc2014a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.777+0000 7fdc24a00640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc201595b0 0x7fdc201f7ef0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.777+0000 7fdc24a00640 1 --2- 172.21.15.82:0/1702799745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc200aee40 0x7fdc20151e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.777+0000 7fdc24a00640 1 -- 172.21.15.82:0/1702799745 >> 172.21.15.82:0/1702799745 conn(0x7fdc2006ae70 msgr2=0x7fdc2015ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:43.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.777+0000 7fdc24a00640 1 -- 172.21.15.82:0/1702799745 shutdown_connections 2024-03-16T18:03:43.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.777+0000 7fdc24a00640 1 -- 172.21.15.82:0/1702799745 wait complete. 2024-03-16T18:03:43.800 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T18:03:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.990+0000 7f0e86400640 1 Processor -- start 2024-03-16T18:03:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.990+0000 7f0e86400640 1 -- start start 2024-03-16T18:03:43.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.991+0000 7f0e86400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0e801454d0 0x7f0e800b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.991+0000 7f0e86400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f0e800cdf80 0x7f0e800cf930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:43.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.991+0000 7f0e86400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e800d02b0 con 0x7f0e800cfe70 2024-03-16T18:03:43.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.991+0000 7f0e86400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e800b4b90 con 0x7f0e801454d0 2024-03-16T18:03:43.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.991+0000 7f0e86400640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e800cf500 con 0x7f0e800cdf80 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7f400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0e801454d0 0x7f0e800b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7f400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0e801454d0 0x7f0e800b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35938/0 (socket says 172.21.15.82:35938) 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7fe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0e800cfe70 0x7f0e8015f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51902/0 (socket says 172.21.15.82:51902) 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7f400640 1 -- 172.21.15.82:0/2176007663 learned_addr learned my addr 172.21.15.82:0/2176007663 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7f400640 1 -- 172.21.15.82:0/2176007663 >> v2:172.21.15.90:3300/0 conn(0x7f0e800cdf80 msgr2=0x7f0e800cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7ea00640 1 --2- 172.21.15.82:0/2176007663 >> v2:172.21.15.90:3300/0 conn(0x7f0e800cdf80 0x7f0e800cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7f400640 1 --2- 172.21.15.82:0/2176007663 >> v2:172.21.15.90:3300/0 conn(0x7f0e800cdf80 0x7f0e800cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7f400640 1 -- 172.21.15.82:0/2176007663 >> v1:172.21.15.82:6789/0 conn(0x7f0e800cfe70 legacy=0x7f0e8015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7f400640 1 -- 172.21.15.82:0/2176007663 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e801458b0 con 0x7f0e801454d0 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7ea00640 1 --2- 172.21.15.82:0/2176007663 >> v2:172.21.15.90:3300/0 conn(0x7f0e800cdf80 0x7f0e800cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:43.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.992+0000 7f0e7e000640 1 -- 172.21.15.82:0/2176007663 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135615973 0 0) 0x7f0e800d02b0 con 0x7f0e800cfe70 2024-03-16T18:03:43.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.993+0000 7f0e7f400640 1 --2- 172.21.15.82:0/2176007663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 0x7f0e800b34c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0e8006ecc0 tx=0x7f0e7002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168189cfcaf79edd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e7e000640 1 -- 172.21.15.82:0/2176007663 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70030860 con 0x7f0e801454d0 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e7e000640 1 -- 172.21.15.82:0/2176007663 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e700309c0 con 0x7f0e801454d0 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e7e000640 1 -- 172.21.15.82:0/2176007663 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70030d50 con 0x7f0e801454d0 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e86400640 1 -- 172.21.15.82:0/2176007663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 msgr2=0x7f0e800b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e86400640 1 --2- 172.21.15.82:0/2176007663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 0x7f0e800b34c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0e8006ecc0 tx=0x7f0e7002b9a0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e86400640 1 -- 172.21.15.82:0/2176007663 shutdown_connections 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e86400640 1 --2- 172.21.15.82:0/2176007663 >> v2:172.21.15.90:3300/0 conn(0x7f0e800cdf80 0x7f0e800cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e86400640 1 --2- 172.21.15.82:0/2176007663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 0x7f0e800b34c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e86400640 1 -- 172.21.15.82:0/2176007663 >> 172.21.15.82:0/2176007663 conn(0x7f0e8006ae70 msgr2=0x7f0e8006b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e86400640 1 -- 172.21.15.82:0/2176007663 shutdown_connections 2024-03-16T18:03:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.994+0000 7f0e86400640 1 -- 172.21.15.82:0/2176007663 wait complete. 2024-03-16T18:03:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.996+0000 7f0e86400640 1 Processor -- start 2024-03-16T18:03:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.996+0000 7f0e86400640 1 -- start start 2024-03-16T18:03:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.997+0000 7f0e86400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e800cdf80 0x7f0e80137d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.997+0000 7f0e86400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e800cfe70 0x7f0e80155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.997+0000 7f0e86400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 0x7f0e80156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.997+0000 7f0e86400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801559f0 con 0x7f0e801454d0 2024-03-16T18:03:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.997+0000 7f0e86400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80156b40 con 0x7f0e800cfe70 2024-03-16T18:03:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.997+0000 7f0e86400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801550c0 con 0x7f0e800cdf80 2024-03-16T18:03:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.997+0000 7f0e7fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 0x7f0e80156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.997+0000 7f0e7fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 0x7f0e80156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35950/0 (socket says 172.21.15.82:35950) 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.997+0000 7f0e7fe00640 1 -- 172.21.15.82:0/2604279100 learned_addr learned my addr 172.21.15.82:0/2604279100 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e7f400640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e800cdf80 0x7f0e80137d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e7ea00640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e800cfe70 0x7f0e80155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e7fe00640 1 -- 172.21.15.82:0/2604279100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e800cdf80 msgr2=0x7f0e80137d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e7fe00640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e800cdf80 0x7f0e80137d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e7fe00640 1 -- 172.21.15.82:0/2604279100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e800cfe70 msgr2=0x7f0e80155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e7fe00640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e800cfe70 0x7f0e80155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e7fe00640 1 -- 172.21.15.82:0/2604279100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e700094d0 con 0x7f0e801454d0 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e7fe00640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 0x7f0e80156510 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c7a0 tx=0x7f0e7800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:44.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e77400640 1 -- 172.21.15.82:0/2604279100 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800a640 con 0x7f0e801454d0 2024-03-16T18:03:44.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e77400640 1 -- 172.21.15.82:0/2604279100 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7800a7a0 con 0x7f0e801454d0 2024-03-16T18:03:44.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e77400640 1 -- 172.21.15.82:0/2604279100 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800abd0 con 0x7f0e801454d0 2024-03-16T18:03:44.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.998+0000 7f0e86400640 1 -- 172.21.15.82:0/2604279100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e801553a0 con 0x7f0e801454d0 2024-03-16T18:03:44.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:43.999+0000 7f0e86400640 1 -- 172.21.15.82:0/2604279100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8015b7a0 con 0x7f0e801454d0 2024-03-16T18:03:44.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.000+0000 7f0e86400640 1 -- 172.21.15.82:0/2604279100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e80067f30 con 0x7f0e801454d0 2024-03-16T18:03:44.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.001+0000 7f0e77400640 1 -- 172.21.15.82:0/2604279100 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f0e7800ad30 con 0x7f0e801454d0 2024-03-16T18:03:44.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.002+0000 7f0e77400640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 0x7f0e6006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.002+0000 7f0e77400640 1 -- 172.21.15.82:0/2604279100 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e78089400 con 0x7f0e801454d0 2024-03-16T18:03:44.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.002+0000 7f0e7f400640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 0x7f0e6006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.003+0000 7f0e7f400640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 0x7f0e6006d870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0e70036000 tx=0x7f0e7003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:44.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.006+0000 7f0e77400640 1 -- 172.21.15.82:0/2604279100 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0e780540f0 con 0x7f0e801454d0 2024-03-16T18:03:44.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.204+0000 7f0e86400640 1 -- 172.21.15.82:0/2604279100 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e80155b30 con 0x7f0e801454d0 2024-03-16T18:03:44.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.207+0000 7f0e77400640 1 -- 172.21.15.82:0/2604279100 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16633 (secure 0 0 0) 0x7f0e780577b0 con 0x7f0e801454d0 2024-03-16T18:03:44.211 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:44.211 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:51.442896+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:58:47.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T17:58:48.893361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T18:03:44.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.209+0000 7f0e86400640 1 -- 172.21.15.82:0/2604279100 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 msgr2=0x7f0e6006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.209+0000 7f0e86400640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 0x7f0e6006d870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0e70036000 tx=0x7f0e7003b040 comp rx=0 tx=0).stop 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.209+0000 7f0e86400640 1 -- 172.21.15.82:0/2604279100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 msgr2=0x7f0e80156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.209+0000 7f0e86400640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 0x7f0e80156510 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c7a0 tx=0x7f0e7800cc60 comp rx=0 tx=0).stop 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.210+0000 7f0e86400640 1 -- 172.21.15.82:0/2604279100 shutdown_connections 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.210+0000 7f0e86400640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f0e6006b3b0 0x7f0e6006d870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.210+0000 7f0e86400640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e801454d0 0x7f0e80156510 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.210+0000 7f0e86400640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e800cfe70 0x7f0e80155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.210+0000 7f0e86400640 1 --2- 172.21.15.82:0/2604279100 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f0e800cdf80 0x7f0e80137d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.210+0000 7f0e86400640 1 -- 172.21.15.82:0/2604279100 >> 172.21.15.82:0/2604279100 conn(0x7f0e8006ae70 msgr2=0x7f0e801422f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.210+0000 7f0e86400640 1 -- 172.21.15.82:0/2604279100 shutdown_connections 2024-03-16T18:03:44.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.210+0000 7f0e86400640 1 -- 172.21.15.82:0/2604279100 wait complete. 2024-03-16T18:03:44.234 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T18:03:44.234 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T18:03:44.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.436+0000 7f7059000640 1 Processor -- start 2024-03-16T18:03:44.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.436+0000 7f7059000640 1 -- start start 2024-03-16T18:03:44.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.437+0000 7f7059000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f70541595b0 0x7f70540ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.437+0000 7f7059000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70540b3370 con 0x7f7054159990 2024-03-16T18:03:44.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.437+0000 7f7059000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705406dd20 con 0x7f705406d740 2024-03-16T18:03:44.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.437+0000 7f7059000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705415f7a0 con 0x7f70541595b0 2024-03-16T18:03:44.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.438+0000 7f7053400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f70541595b0 0x7f70540ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.438+0000 7f7053400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f70541595b0 0x7f70540ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35960/0 (socket says 172.21.15.82:35960) 2024-03-16T18:03:44.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.438+0000 7f7053400640 1 -- 172.21.15.82:0/3009804404 learned_addr learned my addr 172.21.15.82:0/3009804404 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:44.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.438+0000 7f7053400640 1 -- 172.21.15.82:0/3009804404 >> v1:172.21.15.90:6789/0 conn(0x7f705406d740 legacy=0x7f705415c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:44.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.438+0000 7f7053400640 1 -- 172.21.15.82:0/3009804404 >> v1:172.21.15.82:6789/0 conn(0x7f7054159990 legacy=0x7f705406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.438+0000 7f7053400640 1 -- 172.21.15.82:0/3009804404 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705415fa80 con 0x7f70541595b0 2024-03-16T18:03:44.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.439+0000 7f7053400640 1 --2- 172.21.15.82:0/3009804404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70541595b0 0x7f70540ae730 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7044009870 tx=0x7f704402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bef414761cd89c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:44.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.439+0000 7f7052a00640 1 -- 172.21.15.82:0/3009804404 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044030860 con 0x7f70541595b0 2024-03-16T18:03:44.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.439+0000 7f7052a00640 1 -- 172.21.15.82:0/3009804404 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70440309c0 con 0x7f70541595b0 2024-03-16T18:03:44.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.439+0000 7f7052a00640 1 -- 172.21.15.82:0/3009804404 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044030d50 con 0x7f70541595b0 2024-03-16T18:03:44.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.440+0000 7f7059000640 1 -- 172.21.15.82:0/3009804404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70541595b0 msgr2=0x7f70540ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.440+0000 7f7059000640 1 --2- 172.21.15.82:0/3009804404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70541595b0 0x7f70540ae730 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7044009870 tx=0x7f704402b9a0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.440+0000 7f7059000640 1 -- 172.21.15.82:0/3009804404 shutdown_connections 2024-03-16T18:03:44.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.440+0000 7f7059000640 1 --2- 172.21.15.82:0/3009804404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70541595b0 0x7f70540ae730 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.440+0000 7f7059000640 1 -- 172.21.15.82:0/3009804404 >> 172.21.15.82:0/3009804404 conn(0x7f705406ae70 msgr2=0x7f705406b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:44.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.440+0000 7f7059000640 1 -- 172.21.15.82:0/3009804404 shutdown_connections 2024-03-16T18:03:44.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.440+0000 7f7059000640 1 -- 172.21.15.82:0/3009804404 wait complete. 2024-03-16T18:03:44.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.442+0000 7f7059000640 1 Processor -- start 2024-03-16T18:03:44.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.442+0000 7f7059000640 1 -- start start 2024-03-16T18:03:44.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.443+0000 7f7059000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705406d740 0x7f705415dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.443+0000 7f7059000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70541595b0 0x7f7054147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.443+0000 7f7059000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7054159990 0x7f7054147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.443+0000 7f7059000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70541f3fb0 con 0x7f705406d740 2024-03-16T18:03:44.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.443+0000 7f7059000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70541f4db0 con 0x7f70541595b0 2024-03-16T18:03:44.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.443+0000 7f7059000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70541f41a0 con 0x7f7054159990 2024-03-16T18:03:44.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.443+0000 7f7053400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705406d740 0x7f705415dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.443+0000 7f7053400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705406d740 0x7f705415dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35972/0 (socket says 172.21.15.82:35972) 2024-03-16T18:03:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.443+0000 7f7053400640 1 -- 172.21.15.82:0/2350141946 learned_addr learned my addr 172.21.15.82:0/2350141946 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.443+0000 7f704be00640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70541595b0 0x7f7054147120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f7053400640 1 -- 172.21.15.82:0/2350141946 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7054159990 msgr2=0x7f7054147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f7053400640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7054159990 0x7f7054147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f7053400640 1 -- 172.21.15.82:0/2350141946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70541595b0 msgr2=0x7f7054147120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f7053400640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70541595b0 0x7f7054147120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f7053400640 1 -- 172.21.15.82:0/2350141946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70440094d0 con 0x7f705406d740 2024-03-16T18:03:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f704be00640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70541595b0 0x7f7054147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:44.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f7053400640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705406d740 0x7f705415dc70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7044009870 tx=0x7f7044004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:44.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f704b400640 1 -- 172.21.15.82:0/2350141946 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7044039d60 con 0x7f705406d740 2024-03-16T18:03:44.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f704b400640 1 -- 172.21.15.82:0/2350141946 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7044043400 con 0x7f705406d740 2024-03-16T18:03:44.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f704b400640 1 -- 172.21.15.82:0/2350141946 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70440437b0 con 0x7f705406d740 2024-03-16T18:03:44.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f7059000640 1 -- 172.21.15.82:0/2350141946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70541606e0 con 0x7f705406d740 2024-03-16T18:03:44.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.444+0000 7f7059000640 1 -- 172.21.15.82:0/2350141946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70541f4760 con 0x7f705406d740 2024-03-16T18:03:44.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.445+0000 7f7059000640 1 -- 172.21.15.82:0/2350141946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7054154f20 con 0x7f705406d740 2024-03-16T18:03:44.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.446+0000 7f704b400640 1 -- 172.21.15.82:0/2350141946 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f7044043910 con 0x7f705406d740 2024-03-16T18:03:44.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.447+0000 7f704b400640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703006b3b0 0x7f703006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.447+0000 7f704b400640 1 -- 172.21.15.82:0/2350141946 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f704403e070 con 0x7f705406d740 2024-03-16T18:03:44.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.447+0000 7f704be00640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703006b3b0 0x7f703006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.448+0000 7f704be00640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703006b3b0 0x7f703006d870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f705415e790 tx=0x7f703c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:44.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.451+0000 7f704b400640 1 -- 172.21.15.82:0/2350141946 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f70440b3080 con 0x7f705406d740 2024-03-16T18:03:44.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.685+0000 7f7059000640 1 -- 172.21.15.82:0/2350141946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f705404f250 con 0x7f705406d740 2024-03-16T18:03:44.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.686+0000 7f704b400640 1 -- 172.21.15.82:0/2350141946 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4974 (secure 0 0 0) 0x7f7044034090 con 0x7f705406d740 2024-03-16T18:03:44.690 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:44.690 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5100,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3801030113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3801030113},{"type":"v1","addr":"172.21.15.82:6836","nonce":3801030113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2266371611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2266371611},{"type":"v1","addr":"172.21.15.82:6837","nonce":2266371611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/693422260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":693422260},{"type":"v1","addr":"172.21.15.90:6834","nonce":693422260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:58:49.876364+0000","modified":"2024-03-16T17:58:52.961483+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6835/2421794786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":2421794786},{"type":"v1","addr":"172.21.15.90:6835","nonce":2421794786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5118,"qdb_cluster":[5118]},"id":2}]} 2024-03-16T18:03:44.690 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 16 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.688+0000 7f7059000640 1 -- 172.21.15.82:0/2350141946 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703006b3b0 msgr2=0x7f703006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.688+0000 7f7059000640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703006b3b0 0x7f703006d870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f705415e790 tx=0x7f703c009040 comp rx=0 tx=0).stop 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.689+0000 7f7059000640 1 -- 172.21.15.82:0/2350141946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705406d740 msgr2=0x7f705415dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.689+0000 7f7059000640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705406d740 0x7f705415dc70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7044009870 tx=0x7f7044004730 comp rx=0 tx=0).stop 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.689+0000 7f7059000640 1 -- 172.21.15.82:0/2350141946 shutdown_connections 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.689+0000 7f7059000640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f703006b3b0 0x7f703006d870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.689+0000 7f7059000640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7054159990 0x7f7054147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.689+0000 7f7059000640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f70541595b0 0x7f7054147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.689+0000 7f7059000640 1 --2- 172.21.15.82:0/2350141946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705406d740 0x7f705415dc70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.689+0000 7f7059000640 1 -- 172.21.15.82:0/2350141946 >> 172.21.15.82:0/2350141946 conn(0x7f705406ae70 msgr2=0x7f70540ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.689+0000 7f7059000640 1 -- 172.21.15.82:0/2350141946 shutdown_connections 2024-03-16T18:03:44.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.689+0000 7f7059000640 1 -- 172.21.15.82:0/2350141946 wait complete. 2024-03-16T18:03:44.711 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T18:03:44.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.916+0000 7ff723e00640 1 Processor -- start 2024-03-16T18:03:44.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.916+0000 7ff723e00640 1 -- start start 2024-03-16T18:03:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.918+0000 7ff723e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724144480 0x7ff72415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.918+0000 7ff723e00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7240ce670 con 0x7ff72406e250 2024-03-16T18:03:44.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.918+0000 7ff723e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7240ccff0 con 0x7ff7241440a0 2024-03-16T18:03:44.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.918+0000 7ff723e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72415f1a0 con 0x7ff724144480 2024-03-16T18:03:44.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.918+0000 7ff723400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff72406e250 0x7ff72415e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56008/0 (socket says 172.21.15.82:56008) 2024-03-16T18:03:44.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.918+0000 7ff723400640 1 -- 172.21.15.82:0/2396067906 learned_addr learned my addr 172.21.15.82:0/2396067906 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:44.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.918+0000 7ff71a000640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724144480 0x7ff72415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295978201 0 0) 0x7ff7240ce670 con 0x7ff72406e250 2024-03-16T18:03:44.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff708003520 con 0x7ff72406e250 2024-03-16T18:03:44.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958667767 0 0) 0x7ff7240ccff0 con 0x7ff7241440a0 2024-03-16T18:03:44.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 --> v1:172.21.15.90:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7240ce670 con 0x7ff7241440a0 2024-03-16T18:03:44.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1065573867 0 0) 0x7ff708003520 con 0x7ff72406e250 2024-03-16T18:03:44.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724144480 msgr2=0x7ff72415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724144480 0x7ff72415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 >> v1:172.21.15.90:6789/0 conn(0x7ff7241440a0 legacy=0x7ff72406db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240cc230 con 0x7ff72406e250 2024-03-16T18:03:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7ff71c0019f0 con 0x7ff72406e250 2024-03-16T18:03:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 >> v1:172.21.15.82:6789/0 conn(0x7ff72406e250 legacy=0x7ff72415e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff708003ba0 0x7ff708003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.919+0000 7ff722000640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 0x7ff708004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff722000640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724144480 0x7ff7080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72415f1a0 con 0x7ff7080047c0 2024-03-16T18:03:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70800bae0 con 0x7ff708003ba0 2024-03-16T18:03:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70800bca0 con 0x7ff724144480 2024-03-16T18:03:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff71a000640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 0x7ff708004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff722a00640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff708003ba0 0x7ff708003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff71a000640 1 -- 172.21.15.82:0/2396067906 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724144480 msgr2=0x7ff7080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff71a000640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724144480 0x7ff7080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff71a000640 1 -- 172.21.15.82:0/2396067906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff708003ba0 msgr2=0x7ff708003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff71a000640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff708003ba0 0x7ff708003fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.920+0000 7ff71a000640 1 -- 172.21.15.82:0/2396067906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff708003520 con 0x7ff7080047c0 2024-03-16T18:03:44.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.921+0000 7ff71a000640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 0x7ff708004c40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009a90 tx=0x7ff7080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b50cbf27db8cbbd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:44.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.921+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c02c6b0 con 0x7ff7080047c0 2024-03-16T18:03:44.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.921+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70c02c810 con 0x7ff7080047c0 2024-03-16T18:03:44.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.921+0000 7ff722000640 1 -- 172.21.15.82:0/2396067906 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c02cba0 con 0x7ff7080047c0 2024-03-16T18:03:44.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.922+0000 7ff723e00640 1 -- 172.21.15.82:0/2396067906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 msgr2=0x7ff708004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.922+0000 7ff723e00640 1 --2- 172.21.15.82:0/2396067906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 0x7ff708004c40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009a90 tx=0x7ff7080057f0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.922+0000 7ff723400640 1 -- 172.21.15.82:0/2396067906 reap_dead start 2024-03-16T18:03:44.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.922+0000 7ff723e00640 1 -- 172.21.15.82:0/2396067906 shutdown_connections 2024-03-16T18:03:44.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.922+0000 7ff723e00640 1 -- 172.21.15.82:0/2396067906 >> 172.21.15.82:0/2396067906 conn(0x7ff72406ae70 msgr2=0x7ff72406b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:44.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.922+0000 7ff723e00640 1 -- 172.21.15.82:0/2396067906 shutdown_connections 2024-03-16T18:03:44.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.922+0000 7ff723e00640 1 -- 172.21.15.82:0/2396067906 wait complete. 2024-03-16T18:03:44.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.924+0000 7ff723e00640 1 Processor -- start 2024-03-16T18:03:44.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.924+0000 7ff723e00640 1 -- start start 2024-03-16T18:03:44.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff723e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 0x7ff72415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff723e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724162040 0x7ff7240af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff723e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240cfab0 0x7ff7240cff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff723e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240d0480 con 0x7ff7080047c0 2024-03-16T18:03:44.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff723e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240afd40 con 0x7ff7240cfab0 2024-03-16T18:03:44.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff723e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7240b1470 con 0x7ff724162040 2024-03-16T18:03:44.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff722a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 0x7ff72415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff722a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 0x7ff72415a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49060/0 (socket says 172.21.15.82:49060) 2024-03-16T18:03:44.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff722a00640 1 -- 172.21.15.82:0/1150694924 learned_addr learned my addr 172.21.15.82:0/1150694924 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:44.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff71a000640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724162040 0x7ff7240af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff722a00640 1 -- 172.21.15.82:0/1150694924 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724162040 msgr2=0x7ff7240af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff722a00640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724162040 0x7ff7240af460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff723400640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240cfab0 0x7ff7240cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff722a00640 1 -- 172.21.15.82:0/1150694924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240cfab0 msgr2=0x7ff7240cff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:44.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff722a00640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240cfab0 0x7ff7240cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:44.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.925+0000 7ff722a00640 1 -- 172.21.15.82:0/1150694924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240b1750 con 0x7ff7080047c0 2024-03-16T18:03:44.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.926+0000 7ff722a00640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 0x7ff72415a490 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff71400d7a0 tx=0x7ff71400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:44.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.926+0000 7ff71b400640 1 -- 172.21.15.82:0/1150694924 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71400b8b0 con 0x7ff7080047c0 2024-03-16T18:03:44.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.926+0000 7ff71b400640 1 -- 172.21.15.82:0/1150694924 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71400ba10 con 0x7ff7080047c0 2024-03-16T18:03:44.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.926+0000 7ff71b400640 1 -- 172.21.15.82:0/1150694924 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71400bdc0 con 0x7ff7080047c0 2024-03-16T18:03:44.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.926+0000 7ff723e00640 1 -- 172.21.15.82:0/1150694924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7240b1a30 con 0x7ff7080047c0 2024-03-16T18:03:44.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.926+0000 7ff723e00640 1 -- 172.21.15.82:0/1150694924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72415a9d0 con 0x7ff7080047c0 2024-03-16T18:03:44.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.927+0000 7ff723e00640 1 -- 172.21.15.82:0/1150694924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7240b1f50 con 0x7ff7080047c0 2024-03-16T18:03:44.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.928+0000 7ff71b400640 1 -- 172.21.15.82:0/1150694924 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7ff7140123f0 con 0x7ff7080047c0 2024-03-16T18:03:44.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.929+0000 7ff71b400640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff70006b3b0 0x7ff70006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:44.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.929+0000 7ff71b400640 1 -- 172.21.15.82:0/1150694924 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff714089ed0 con 0x7ff7080047c0 2024-03-16T18:03:44.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.929+0000 7ff71a000640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff70006b3b0 0x7ff70006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:44.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.930+0000 7ff71a000640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff70006b3b0 0x7ff70006d870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff70c036000 tx=0x7ff70c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:44.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:44.933+0000 7ff71b400640 1 -- 172.21.15.82:0/1150694924 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff7240b1f50 con 0x7ff7080047c0 2024-03-16T18:03:45.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.167+0000 7ff723e00640 1 -- 172.21.15.82:0/1150694924 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7240b1f50 con 0x7ff7080047c0 2024-03-16T18:03:45.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.168+0000 7ff71b400640 1 -- 172.21.15.82:0/1150694924 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4974 (secure 0 0 0) 0x7ff7240b1f50 con 0x7ff7080047c0 2024-03-16T18:03:45.172 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:45.172 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5100,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3801030113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3801030113},{"type":"v1","addr":"172.21.15.82:6836","nonce":3801030113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2266371611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2266371611},{"type":"v1","addr":"172.21.15.82:6837","nonce":2266371611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/693422260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":693422260},{"type":"v1","addr":"172.21.15.90:6834","nonce":693422260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T17:58:49.876364+0000","modified":"2024-03-16T17:58:52.961483+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5118},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5118":{"gid":5118,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.90:6835/2421794786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":2421794786},{"type":"v1","addr":"172.21.15.90:6835","nonce":2421794786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5118,"qdb_cluster":[5118]},"id":2}]} 2024-03-16T18:03:45.172 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 16 2024-03-16T18:03:45.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.171+0000 7ff723e00640 1 -- 172.21.15.82:0/1150694924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff70006b3b0 msgr2=0x7ff70006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:45.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.171+0000 7ff723e00640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff70006b3b0 0x7ff70006d870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff70c036000 tx=0x7ff70c030000 comp rx=0 tx=0).stop 2024-03-16T18:03:45.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.171+0000 7ff723e00640 1 -- 172.21.15.82:0/1150694924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 msgr2=0x7ff72415a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:45.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.171+0000 7ff723e00640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 0x7ff72415a490 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff71400d7a0 tx=0x7ff71400dc60 comp rx=0 tx=0).stop 2024-03-16T18:03:45.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.172+0000 7ff723e00640 1 -- 172.21.15.82:0/1150694924 shutdown_connections 2024-03-16T18:03:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.172+0000 7ff723e00640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff70006b3b0 0x7ff70006d870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.172+0000 7ff723e00640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240cfab0 0x7ff7240cff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.172+0000 7ff723e00640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff724162040 0x7ff7240af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.172+0000 7ff723e00640 1 --2- 172.21.15.82:0/1150694924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff7080047c0 0x7ff72415a490 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.172+0000 7ff723e00640 1 -- 172.21.15.82:0/1150694924 >> 172.21.15.82:0/1150694924 conn(0x7ff72406ae70 msgr2=0x7ff72415c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.172+0000 7ff723e00640 1 -- 172.21.15.82:0/1150694924 shutdown_connections 2024-03-16T18:03:45.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.172+0000 7ff723e00640 1 -- 172.21.15.82:0/1150694924 wait complete. 2024-03-16T18:03:45.192 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T18:03:45.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.394+0000 7f031a200640 1 Processor -- start 2024-03-16T18:03:45.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.394+0000 7f031a200640 1 -- start start 2024-03-16T18:03:45.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.395+0000 7f031a200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031415df00 0x7f03140d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.395+0000 7f031a200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f03140b32c0 0x7f03140d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.395+0000 7f031a200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f031415e2e0 0x7f03140d28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.395+0000 7f031a200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03140cfe40 con 0x7f031415e2e0 2024-03-16T18:03:45.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.395+0000 7f031a200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03140d4940 con 0x7f03140b32c0 2024-03-16T18:03:45.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.395+0000 7f031a200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03140b3780 con 0x7f031415df00 2024-03-16T18:03:45.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.396+0000 7f0313e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f031415e2e0 0x7f03140d28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.396+0000 7f0313e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f031415e2e0 0x7f03140d28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49072/0 (socket says 172.21.15.82:49072) 2024-03-16T18:03:45.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.396+0000 7f0313e00640 1 -- 172.21.15.82:0/231972634 learned_addr learned my addr 172.21.15.82:0/231972634 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:45.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.396+0000 7f0313400640 1 --2- 172.21.15.82:0/231972634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031415df00 0x7f03140d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.396+0000 7f0312a00640 1 --2- 172.21.15.82:0/231972634 >> v2:172.21.15.90:3300/0 conn(0x7f03140b32c0 0x7f03140d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.396+0000 7f0313e00640 1 -- 172.21.15.82:0/231972634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031415df00 msgr2=0x7f03140d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:45.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.396+0000 7f0313e00640 1 --2- 172.21.15.82:0/231972634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031415df00 0x7f03140d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.396+0000 7f0313e00640 1 -- 172.21.15.82:0/231972634 >> v2:172.21.15.90:3300/0 conn(0x7f03140b32c0 msgr2=0x7f03140d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:45.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.396+0000 7f0313e00640 1 --2- 172.21.15.82:0/231972634 >> v2:172.21.15.90:3300/0 conn(0x7f03140b32c0 0x7f03140d2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.396+0000 7f0313e00640 1 -- 172.21.15.82:0/231972634 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03140b3a60 con 0x7f031415e2e0 2024-03-16T18:03:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.398+0000 7f0313e00640 1 --2- 172.21.15.82:0/231972634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 0x7f03140d28a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f031408d760 tx=0x7f030c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d9417ab35142c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.398+0000 7f0312000640 1 -- 172.21.15.82:0/231972634 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c02cb00 con 0x7f031415e2e0 2024-03-16T18:03:45.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.398+0000 7f0312000640 1 -- 172.21.15.82:0/231972634 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c02cc60 con 0x7f031415e2e0 2024-03-16T18:03:45.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.398+0000 7f0312000640 1 -- 172.21.15.82:0/231972634 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c0343f0 con 0x7f031415e2e0 2024-03-16T18:03:45.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.399+0000 7f031a200640 1 -- 172.21.15.82:0/231972634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 msgr2=0x7f03140d28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:45.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.399+0000 7f031a200640 1 --2- 172.21.15.82:0/231972634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 0x7f03140d28a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f031408d760 tx=0x7f030c02bf60 comp rx=0 tx=0).stop 2024-03-16T18:03:45.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.399+0000 7f031a200640 1 -- 172.21.15.82:0/231972634 shutdown_connections 2024-03-16T18:03:45.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.399+0000 7f031a200640 1 --2- 172.21.15.82:0/231972634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 0x7f03140d28a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.399+0000 7f031a200640 1 --2- 172.21.15.82:0/231972634 >> v2:172.21.15.90:3300/0 conn(0x7f03140b32c0 0x7f03140d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.399+0000 7f031a200640 1 --2- 172.21.15.82:0/231972634 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f031415df00 0x7f03140d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.399+0000 7f031a200640 1 -- 172.21.15.82:0/231972634 >> 172.21.15.82:0/231972634 conn(0x7f03140d3d80 msgr2=0x7f03140d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.399+0000 7f031a200640 1 -- 172.21.15.82:0/231972634 shutdown_connections 2024-03-16T18:03:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.399+0000 7f031a200640 1 -- 172.21.15.82:0/231972634 wait complete. 2024-03-16T18:03:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.401+0000 7f031a200640 1 Processor -- start 2024-03-16T18:03:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.401+0000 7f031a200640 1 -- start start 2024-03-16T18:03:45.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.402+0000 7f031a200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f03140b32c0 0x7f03141f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.402+0000 7f031a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f031415df00 0x7f0314160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.402+0000 7f031a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 0x7f0314160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.402+0000 7f031a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03141f6f60 con 0x7f031415e2e0 2024-03-16T18:03:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.402+0000 7f031a200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03140cdf90 con 0x7f031415df00 2024-03-16T18:03:45.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.402+0000 7f031a200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03140ccc50 con 0x7f03140b32c0 2024-03-16T18:03:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.402+0000 7f0313e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 0x7f0314160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.402+0000 7f0313e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 0x7f0314160c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49078/0 (socket says 172.21.15.82:49078) 2024-03-16T18:03:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.402+0000 7f0313e00640 1 -- 172.21.15.82:0/66854026 learned_addr learned my addr 172.21.15.82:0/66854026 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.403+0000 7f0313e00640 1 -- 172.21.15.82:0/66854026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f03140b32c0 msgr2=0x7f03141f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.403+0000 7f0313e00640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f03140b32c0 0x7f03141f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.403+0000 7f0313e00640 1 -- 172.21.15.82:0/66854026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f031415df00 msgr2=0x7f0314160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.403+0000 7f0313e00640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f031415df00 0x7f0314160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.403+0000 7f0313e00640 1 -- 172.21.15.82:0/66854026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c009b30 con 0x7f031415e2e0 2024-03-16T18:03:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.403+0000 7f0313e00640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 0x7f0314160c40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f030c036040 tx=0x7f030c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.403+0000 7f030b400640 1 -- 172.21.15.82:0/66854026 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c040670 con 0x7f031415e2e0 2024-03-16T18:03:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.403+0000 7f030b400640 1 -- 172.21.15.82:0/66854026 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030c0407d0 con 0x7f031415e2e0 2024-03-16T18:03:45.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.403+0000 7f031a200640 1 -- 172.21.15.82:0/66854026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03140ccef0 con 0x7f031415e2e0 2024-03-16T18:03:45.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.403+0000 7f030b400640 1 -- 172.21.15.82:0/66854026 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030c040b80 con 0x7f031415e2e0 2024-03-16T18:03:45.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.404+0000 7f031a200640 1 -- 172.21.15.82:0/66854026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03140cd4b0 con 0x7f031415e2e0 2024-03-16T18:03:45.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.405+0000 7f031a200640 1 -- 172.21.15.82:0/66854026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03140d0d40 con 0x7f031415e2e0 2024-03-16T18:03:45.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.406+0000 7f030b400640 1 -- 172.21.15.82:0/66854026 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f030c040ce0 con 0x7f031415e2e0 2024-03-16T18:03:45.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.406+0000 7f030b400640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f02f406b3b0 0x7f02f406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.407+0000 7f030b400640 1 -- 172.21.15.82:0/66854026 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f030c0acfe0 con 0x7f031415e2e0 2024-03-16T18:03:45.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.407+0000 7f0313400640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f02f406b3b0 0x7f02f406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.407+0000 7f0313400640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f02f406b3b0 0x7f02f406d870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0304009870 tx=0x7f0304008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:45.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.411+0000 7f030b400640 1 -- 172.21.15.82:0/66854026 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f030c077be0 con 0x7f031415e2e0 2024-03-16T18:03:45.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.611+0000 7f031a200640 1 -- 172.21.15.82:0/66854026 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03140d3570 con 0x7f031415e2e0 2024-03-16T18:03:45.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.613+0000 7f030b400640 1 -- 172.21.15.82:0/66854026 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16633 (secure 0 0 0) 0x7f030c009510 con 0x7f031415e2e0 2024-03-16T18:03:45.617 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:45.617 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T17:58:51.442896+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T17:58:47.770770+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T17:58:48.893361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 -- 172.21.15.82:0/66854026 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f02f406b3b0 msgr2=0x7f02f406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f02f406b3b0 0x7f02f406d870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0304009870 tx=0x7f0304008040 comp rx=0 tx=0).stop 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 -- 172.21.15.82:0/66854026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 msgr2=0x7f0314160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 0x7f0314160c40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f030c036040 tx=0x7f030c004a40 comp rx=0 tx=0).stop 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 -- 172.21.15.82:0/66854026 shutdown_connections 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f02f406b3b0 0x7f02f406d870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f031415e2e0 0x7f0314160c40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f031415df00 0x7f0314160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 --2- 172.21.15.82:0/66854026 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f03140b32c0 0x7f03141f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 -- 172.21.15.82:0/66854026 >> 172.21.15.82:0/66854026 conn(0x7f03140d3d80 msgr2=0x7f03140cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 -- 172.21.15.82:0/66854026 shutdown_connections 2024-03-16T18:03:45.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.616+0000 7f031a200640 1 -- 172.21.15.82:0/66854026 wait complete. 2024-03-16T18:03:45.639 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-16T18:03:45.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.846+0000 7f2f33e00640 1 Processor -- start 2024-03-16T18:03:45.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.846+0000 7f2f33e00640 1 -- start start 2024-03-16T18:03:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.848+0000 7f2f33e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2f340d04f0 0x7f2f340b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.848+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340cfad0 0x7f2f340d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.848+0000 7f2f33e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f340d42d0 con 0x7f2f340d0e10 2024-03-16T18:03:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.848+0000 7f2f33e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f340cff50 con 0x7f2f340d04f0 2024-03-16T18:03:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.848+0000 7f2f33e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3415e6e0 con 0x7f2f340cfad0 2024-03-16T18:03:45.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.848+0000 7f2f32a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2f340d04f0 0x7f2f340b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.848+0000 7f2f32a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2f340d04f0 0x7f2f340b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49088/0 (socket says 172.21.15.82:49088) 2024-03-16T18:03:45.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.848+0000 7f2f32a00640 1 -- 172.21.15.82:0/453188723 learned_addr learned my addr 172.21.15.82:0/453188723 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:45.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.849+0000 7f2f32000640 1 --2- 172.21.15.82:0/453188723 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340cfad0 0x7f2f340d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.849+0000 7f2f32a00640 1 -- 172.21.15.82:0/453188723 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340cfad0 msgr2=0x7f2f340d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:45.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.849+0000 7f2f32a00640 1 --2- 172.21.15.82:0/453188723 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340cfad0 0x7f2f340d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.849+0000 7f2f32a00640 1 -- 172.21.15.82:0/453188723 >> v1:172.21.15.90:6789/0 conn(0x7f2f340d0e10 legacy=0x7f2f340b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:45.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.849+0000 7f2f32a00640 1 -- 172.21.15.82:0/453188723 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f340d4520 con 0x7f2f340d04f0 2024-03-16T18:03:45.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.849+0000 7f2f32000640 1 --2- 172.21.15.82:0/453188723 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340cfad0 0x7f2f340d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:45.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.850+0000 7f2f32a00640 1 --2- 172.21.15.82:0/453188723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340d04f0 0x7f2f340b5050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009870 tx=0x7f2f2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877818999d51895 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:45.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.850+0000 7f2f31600640 1 -- 172.21.15.82:0/453188723 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24031640 con 0x7f2f340d04f0 2024-03-16T18:03:45.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.850+0000 7f2f31600640 1 -- 172.21.15.82:0/453188723 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f240317a0 con 0x7f2f340d04f0 2024-03-16T18:03:45.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.850+0000 7f2f31600640 1 -- 172.21.15.82:0/453188723 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24031b30 con 0x7f2f340d04f0 2024-03-16T18:03:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.851+0000 7f2f33e00640 1 -- 172.21.15.82:0/453188723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340d04f0 msgr2=0x7f2f340b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.851+0000 7f2f33e00640 1 --2- 172.21.15.82:0/453188723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340d04f0 0x7f2f340b5050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009870 tx=0x7f2f2402b8f0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.851+0000 7f2f33e00640 1 -- 172.21.15.82:0/453188723 shutdown_connections 2024-03-16T18:03:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.851+0000 7f2f33e00640 1 --2- 172.21.15.82:0/453188723 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340cfad0 0x7f2f340d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.851+0000 7f2f33e00640 1 --2- 172.21.15.82:0/453188723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340d04f0 0x7f2f340b5050 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.851+0000 7f2f33e00640 1 -- 172.21.15.82:0/453188723 >> 172.21.15.82:0/453188723 conn(0x7f2f3409cf30 msgr2=0x7f2f340d3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.851+0000 7f2f33e00640 1 -- 172.21.15.82:0/453188723 shutdown_connections 2024-03-16T18:03:45.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.851+0000 7f2f33e00640 1 -- 172.21.15.82:0/453188723 wait complete. 2024-03-16T18:03:45.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.853+0000 7f2f33e00640 1 Processor -- start 2024-03-16T18:03:45.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.853+0000 7f2f33e00640 1 -- start start 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.854+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340cfad0 0x7f2f34152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.854+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340d04f0 0x7f2f340cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.854+0000 7f2f33e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f340d0e10 0x7f2f340b8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.854+0000 7f2f33e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340cc410 con 0x7f2f340cfad0 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.854+0000 7f2f33e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340cc830 con 0x7f2f340d0e10 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.854+0000 7f2f33e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340cb7d0 con 0x7f2f340d04f0 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.854+0000 7f2f32a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340cfad0 0x7f2f34152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.854+0000 7f2f32a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340cfad0 0x7f2f34152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49096/0 (socket says 172.21.15.82:49096) 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.854+0000 7f2f32a00640 1 -- 172.21.15.82:0/2310309938 learned_addr learned my addr 172.21.15.82:0/2310309938 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f32000640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340d04f0 0x7f2f340cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f33400640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f340d0e10 0x7f2f340b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f32a00640 1 -- 172.21.15.82:0/2310309938 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340d04f0 msgr2=0x7f2f340cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:45.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f32a00640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340d04f0 0x7f2f340cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f32a00640 1 -- 172.21.15.82:0/2310309938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f340d0e10 msgr2=0x7f2f340b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f32a00640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f340d0e10 0x7f2f340b8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f32a00640 1 -- 172.21.15.82:0/2310309938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f240094d0 con 0x7f2f340cfad0 2024-03-16T18:03:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f32a00640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340cfad0 0x7f2f34152f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2f24000c00 tx=0x7f2f240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f2aa00640 1 -- 172.21.15.82:0/2310309938 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2403ac60 con 0x7f2f340cfad0 2024-03-16T18:03:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f2aa00640 1 -- 172.21.15.82:0/2310309938 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2403ade0 con 0x7f2f340cfad0 2024-03-16T18:03:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f2aa00640 1 -- 172.21.15.82:0/2310309938 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f24044630 con 0x7f2f340cfad0 2024-03-16T18:03:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.855+0000 7f2f33e00640 1 -- 172.21.15.82:0/2310309938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f340cba70 con 0x7f2f340cfad0 2024-03-16T18:03:45.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.856+0000 7f2f33e00640 1 -- 172.21.15.82:0/2310309938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f340cc030 con 0x7f2f340cfad0 2024-03-16T18:03:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.857+0000 7f2f33e00640 1 -- 172.21.15.82:0/2310309938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f340d15e0 con 0x7f2f340cfad0 2024-03-16T18:03:45.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.858+0000 7f2f2aa00640 1 -- 172.21.15.82:0/2310309938 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f2f2402f050 con 0x7f2f340cfad0 2024-03-16T18:03:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.859+0000 7f2f2aa00640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f1006b3b0 0x7f2f1006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.859+0000 7f2f2aa00640 1 -- 172.21.15.82:0/2310309938 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f240abc50 con 0x7f2f340cfad0 2024-03-16T18:03:45.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.859+0000 7f2f32000640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f1006b3b0 0x7f2f1006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:45.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.860+0000 7f2f32000640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f1006b3b0 0x7f2f1006d870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f34068420 tx=0x7f2f1c008450 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:45.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:45.863+0000 7f2f2aa00640 1 -- 172.21.15.82:0/2310309938 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2f240b1020 con 0x7f2f340cfad0 2024-03-16T18:03:46.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.093+0000 7f2f33e00640 1 -- 172.21.15.82:0/2310309938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2f341f4fc0 con 0x7f2f340cfad0 2024-03-16T18:03:46.097 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-03-16T18:03:46.093+0000 7f1de0600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T18:03:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.097+0000 7f2f2aa00640 1 -- 172.21.15.82:0/2310309938 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2f2407a470 con 0x7f2f340cfad0 2024-03-16T18:03:46.100 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-16T18:03:46.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.101+0000 7f2f33e00640 1 -- 172.21.15.82:0/2310309938 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f1006b3b0 msgr2=0x7f2f1006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:46.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.101+0000 7f2f33e00640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f1006b3b0 0x7f2f1006d870 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f34068420 tx=0x7f2f1c008450 comp rx=0 tx=0).stop 2024-03-16T18:03:46.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.101+0000 7f2f33e00640 1 -- 172.21.15.82:0/2310309938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340cfad0 msgr2=0x7f2f34152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:46.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.101+0000 7f2f33e00640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340cfad0 0x7f2f34152f20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2f24000c00 tx=0x7f2f240046c0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.102+0000 7f2f33e00640 1 -- 172.21.15.82:0/2310309938 shutdown_connections 2024-03-16T18:03:46.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.102+0000 7f2f33e00640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f2f1006b3b0 0x7f2f1006d870 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.102+0000 7f2f33e00640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2f340d0e10 0x7f2f340b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.102+0000 7f2f33e00640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f2f340d04f0 0x7f2f340cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.102+0000 7f2f33e00640 1 --2- 172.21.15.82:0/2310309938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f340cfad0 0x7f2f34152f20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.102+0000 7f2f33e00640 1 -- 172.21.15.82:0/2310309938 >> 172.21.15.82:0/2310309938 conn(0x7f2f3409cf30 msgr2=0x7f2f3415ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:46.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.102+0000 7f2f33e00640 1 -- 172.21.15.82:0/2310309938 shutdown_connections 2024-03-16T18:03:46.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.102+0000 7f2f33e00640 1 -- 172.21.15.82:0/2310309938 wait complete. 2024-03-16T18:03:46.126 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-16T18:03:46.231 INFO:tasks.ceph.mds.b.smithi090.stdout:starting mds.b at 2024-03-16T18:03:46.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.326+0000 7fb656600640 1 Processor -- start 2024-03-16T18:03:46.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.326+0000 7fb656600640 1 -- start start 2024-03-16T18:03:46.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.327+0000 7fb656600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb650159960 0x7fb65006e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:46.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.327+0000 7fb656600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb65006e770 0x7fb6500cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:46.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.328+0000 7fb656600640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6500ce550 con 0x7fb6500cc850 2024-03-16T18:03:46.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.328+0000 7fb656600640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb650159d40 con 0x7fb65006e770 2024-03-16T18:03:46.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.328+0000 7fb656600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65015cff0 con 0x7fb650159960 2024-03-16T18:03:46.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.328+0000 7fb646a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb65006e770 0x7fb6500cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:46.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.328+0000 7fb64f400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb650159960 0x7fb65006e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:46.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.328+0000 7fb646a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fb65006e770 0x7fb6500cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:45416/0 (socket says 172.21.15.82:45416) 2024-03-16T18:03:46.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.328+0000 7fb646a00640 1 -- 172.21.15.82:0/1167769734 learned_addr learned my addr 172.21.15.82:0/1167769734 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:46.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.329+0000 7fb646a00640 1 -- 172.21.15.82:0/1167769734 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb650159960 msgr2=0x7fb65006e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:46.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.329+0000 7fb646a00640 1 --2- 172.21.15.82:0/1167769734 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb650159960 0x7fb65006e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.329+0000 7fb646a00640 1 -- 172.21.15.82:0/1167769734 >> v1:172.21.15.90:6789/0 conn(0x7fb6500cc850 legacy=0x7fb65015f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:46.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.329+0000 7fb646a00640 1 -- 172.21.15.82:0/1167769734 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65015c370 con 0x7fb65006e770 2024-03-16T18:03:46.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.329+0000 7fb64f400640 1 --2- 172.21.15.82:0/1167769734 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb650159960 0x7fb65006e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T18:03:46.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.330+0000 7fb646a00640 1 --2- 172.21.15.82:0/1167769734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb65006e770 0x7fb6500cc310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb638009870 tx=0x7fb63802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=606b066e233bd0f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:46.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.331+0000 7fb64ea00640 1 -- 172.21.15.82:0/1167769734 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638031640 con 0x7fb65006e770 2024-03-16T18:03:46.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.331+0000 7fb64ea00640 1 -- 172.21.15.82:0/1167769734 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6380317a0 con 0x7fb65006e770 2024-03-16T18:03:46.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.331+0000 7fb64ea00640 1 -- 172.21.15.82:0/1167769734 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638031b30 con 0x7fb65006e770 2024-03-16T18:03:46.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.331+0000 7fb656600640 1 -- 172.21.15.82:0/1167769734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb65006e770 msgr2=0x7fb6500cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:46.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.331+0000 7fb656600640 1 --2- 172.21.15.82:0/1167769734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb65006e770 0x7fb6500cc310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb638009870 tx=0x7fb63802b8f0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.331+0000 7fb656600640 1 -- 172.21.15.82:0/1167769734 shutdown_connections 2024-03-16T18:03:46.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.331+0000 7fb656600640 1 --2- 172.21.15.82:0/1167769734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb65006e770 0x7fb6500cc310 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.331+0000 7fb656600640 1 --2- 172.21.15.82:0/1167769734 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb650159960 0x7fb65006e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.331+0000 7fb656600640 1 -- 172.21.15.82:0/1167769734 >> 172.21.15.82:0/1167769734 conn(0x7fb6500b1640 msgr2=0x7fb6500b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:46.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.331+0000 7fb656600640 1 -- 172.21.15.82:0/1167769734 shutdown_connections 2024-03-16T18:03:46.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.332+0000 7fb656600640 1 -- 172.21.15.82:0/1167769734 wait complete. 2024-03-16T18:03:46.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.333+0000 7fb656600640 1 Processor -- start 2024-03-16T18:03:46.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.333+0000 7fb656600640 1 -- start start 2024-03-16T18:03:46.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.334+0000 7fb656600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb65006e770 0x7fb650140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:46.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.334+0000 7fb656600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6500cc850 0x7fb650142830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:46.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.334+0000 7fb656600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb650159960 0x7fb65015e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:46.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.334+0000 7fb656600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb650140990 con 0x7fb6500cc850 2024-03-16T18:03:46.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.334+0000 7fb656600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500b3490 con 0x7fb650159960 2024-03-16T18:03:46.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.334+0000 7fb656600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb650142d70 con 0x7fb65006e770 2024-03-16T18:03:46.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.335+0000 7fb646a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6500cc850 0x7fb650142830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:46.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.335+0000 7fb646a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6500cc850 0x7fb650142830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49112/0 (socket says 172.21.15.82:49112) 2024-03-16T18:03:46.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.335+0000 7fb646a00640 1 -- 172.21.15.82:0/3955369140 learned_addr learned my addr 172.21.15.82:0/3955369140 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:46.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.335+0000 7fb64f400640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb65006e770 0x7fb650140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:46.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.335+0000 7fb64fe00640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb650159960 0x7fb65015e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:46.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.335+0000 7fb646a00640 1 -- 172.21.15.82:0/3955369140 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb65006e770 msgr2=0x7fb650140450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:46.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.335+0000 7fb646a00640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb65006e770 0x7fb650140450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.335+0000 7fb646a00640 1 -- 172.21.15.82:0/3955369140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb650159960 msgr2=0x7fb65015e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:46.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.335+0000 7fb646a00640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb650159960 0x7fb65015e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:46.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.335+0000 7fb646a00640 1 -- 172.21.15.82:0/3955369140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb640009400 con 0x7fb6500cc850 2024-03-16T18:03:46.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.336+0000 7fb646a00640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6500cc850 0x7fb650142830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb638000c00 tx=0x7fb6380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.336+0000 7fb647e00640 1 -- 172.21.15.82:0/3955369140 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63803a930 con 0x7fb6500cc850 2024-03-16T18:03:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.336+0000 7fb647e00640 1 -- 172.21.15.82:0/3955369140 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63803aa90 con 0x7fb6500cc850 2024-03-16T18:03:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.336+0000 7fb656600640 1 -- 172.21.15.82:0/3955369140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6380094d0 con 0x7fb6500cc850 2024-03-16T18:03:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.336+0000 7fb647e00640 1 -- 172.21.15.82:0/3955369140 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63803ae40 con 0x7fb6500cc850 2024-03-16T18:03:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.336+0000 7fb656600640 1 -- 172.21.15.82:0/3955369140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb650143320 con 0x7fb6500cc850 2024-03-16T18:03:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.337+0000 7fb656600640 1 -- 172.21.15.82:0/3955369140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6500b5950 con 0x7fb6500cc850 2024-03-16T18:03:46.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.338+0000 7fb647e00640 1 -- 172.21.15.82:0/3955369140 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fb63802f050 con 0x7fb6500cc850 2024-03-16T18:03:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.339+0000 7fb647e00640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb62c06b3b0 0x7fb62c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.339+0000 7fb647e00640 1 -- 172.21.15.82:0/3955369140 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb6380abad0 con 0x7fb6500cc850 2024-03-16T18:03:46.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.339+0000 7fb64f400640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb62c06b3b0 0x7fb62c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:46.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.340+0000 7fb64f400640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb62c06b3b0 0x7fb62c06d870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb640009c60 tx=0x7fb640008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:46.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.344+0000 7fb647e00640 1 -- 172.21.15.82:0/3955369140 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb638076630 con 0x7fb6500cc850 2024-03-16T18:03:46.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:46.582+0000 7fb656600640 1 -- 172.21.15.82:0/3955369140 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb650067f30 con 0x7fb6500cc850 2024-03-16T18:03:47.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.107+0000 7fb647e00640 1 -- 172.21.15.82:0/3955369140 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb638079cf0 con 0x7fb6500cc850 2024-03-16T18:03:47.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.110+0000 7fb656600640 1 -- 172.21.15.82:0/3955369140 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb62c06b3b0 msgr2=0x7fb62c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:47.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb62c06b3b0 0x7fb62c06d870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb640009c60 tx=0x7fb640008040 comp rx=0 tx=0).stop 2024-03-16T18:03:47.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 -- 172.21.15.82:0/3955369140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6500cc850 msgr2=0x7fb650142830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:47.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6500cc850 0x7fb650142830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb638000c00 tx=0x7fb6380046c0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 -- 172.21.15.82:0/3955369140 shutdown_connections 2024-03-16T18:03:47.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb62c06b3b0 0x7fb62c06d870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb650159960 0x7fb65015e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6500cc850 0x7fb650142830 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 --2- 172.21.15.82:0/3955369140 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb65006e770 0x7fb650140450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 -- 172.21.15.82:0/3955369140 >> 172.21.15.82:0/3955369140 conn(0x7fb6500b1640 msgr2=0x7fb65015f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:47.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 -- 172.21.15.82:0/3955369140 shutdown_connections 2024-03-16T18:03:47.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.111+0000 7fb656600640 1 -- 172.21.15.82:0/3955369140 wait complete. 2024-03-16T18:03:47.137 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-16T18:03:47.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.346+0000 7f835a200640 1 Processor -- start 2024-03-16T18:03:47.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.346+0000 7f835a200640 1 -- start start 2024-03-16T18:03:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f835a200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f835406dc70 0x7f835406e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f835a200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f835415e150 0x7f83541624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f835a200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83540af050 0x7f83540ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f835a200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83540cf720 con 0x7f835415e150 2024-03-16T18:03:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f835a200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f835406e590 con 0x7f835406dc70 2024-03-16T18:03:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f835a200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8354162a30 con 0x7f83540af050 2024-03-16T18:03:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f8352a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f835415e150 0x7f83541624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f8352a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f835415e150 0x7f83541624f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49128/0 (socket says 172.21.15.82:49128) 2024-03-16T18:03:47.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f8352a00640 1 -- 172.21.15.82:0/2741613952 learned_addr learned my addr 172.21.15.82:0/2741613952 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:47.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f8353e00640 1 --2- 172.21.15.82:0/2741613952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83540af050 0x7f83540ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:47.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.348+0000 7f8353400640 1 --2- 172.21.15.82:0/2741613952 >> v2:172.21.15.90:3300/0 conn(0x7f835406dc70 0x7f835406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:47.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.349+0000 7f8352a00640 1 -- 172.21.15.82:0/2741613952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83540af050 msgr2=0x7f83540ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:47.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.349+0000 7f8352a00640 1 --2- 172.21.15.82:0/2741613952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83540af050 0x7f83540ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.349+0000 7f8352a00640 1 -- 172.21.15.82:0/2741613952 >> v2:172.21.15.90:3300/0 conn(0x7f835406dc70 msgr2=0x7f835406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:47.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.349+0000 7f8352a00640 1 --2- 172.21.15.82:0/2741613952 >> v2:172.21.15.90:3300/0 conn(0x7f835406dc70 0x7f835406e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.349+0000 7f8352a00640 1 -- 172.21.15.82:0/2741613952 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83540cc530 con 0x7f835415e150 2024-03-16T18:03:47.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.349+0000 7f8353400640 1 --2- 172.21.15.82:0/2741613952 >> v2:172.21.15.90:3300/0 conn(0x7f835406dc70 0x7f835406e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:47.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.350+0000 7f8352a00640 1 --2- 172.21.15.82:0/2741613952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835415e150 0x7f83541624f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f835406be20 tx=0x7f834002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d0d65c8a1172f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:47.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.350+0000 7f8352000640 1 -- 172.21.15.82:0/2741613952 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83400317d0 con 0x7f835415e150 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.350+0000 7f8352000640 1 -- 172.21.15.82:0/2741613952 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8340031930 con 0x7f835415e150 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.350+0000 7f8352000640 1 -- 172.21.15.82:0/2741613952 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340031cc0 con 0x7f835415e150 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.351+0000 7f835a200640 1 -- 172.21.15.82:0/2741613952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835415e150 msgr2=0x7f83541624f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.351+0000 7f835a200640 1 --2- 172.21.15.82:0/2741613952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835415e150 0x7f83541624f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f835406be20 tx=0x7f834002b870 comp rx=0 tx=0).stop 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.351+0000 7f835a200640 1 -- 172.21.15.82:0/2741613952 shutdown_connections 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.351+0000 7f835a200640 1 --2- 172.21.15.82:0/2741613952 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f83540af050 0x7f83540ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.351+0000 7f835a200640 1 --2- 172.21.15.82:0/2741613952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835415e150 0x7f83541624f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.351+0000 7f835a200640 1 --2- 172.21.15.82:0/2741613952 >> v2:172.21.15.90:3300/0 conn(0x7f835406dc70 0x7f835406e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.351+0000 7f835a200640 1 -- 172.21.15.82:0/2741613952 >> 172.21.15.82:0/2741613952 conn(0x7f83540b1740 msgr2=0x7f83540b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.351+0000 7f835a200640 1 -- 172.21.15.82:0/2741613952 shutdown_connections 2024-03-16T18:03:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.351+0000 7f835a200640 1 -- 172.21.15.82:0/2741613952 wait complete. 2024-03-16T18:03:47.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.352+0000 7f835a200640 1 Processor -- start 2024-03-16T18:03:47.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.353+0000 7f835a200640 1 -- start start 2024-03-16T18:03:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.353+0000 7f835a200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835406dc70 0x7f8354157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.353+0000 7f835a200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83540af050 0x7f83541494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.353+0000 7f835a200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f835415e150 0x7f8354160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.353+0000 7f835a200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8354157ab0 con 0x7f835406dc70 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.353+0000 7f835a200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835415f980 con 0x7f83540af050 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.353+0000 7f835a200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835415fc60 con 0x7f835415e150 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.354+0000 7f8353400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835406dc70 0x7f8354157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.354+0000 7f8353400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835406dc70 0x7f8354157570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49130/0 (socket says 172.21.15.82:49130) 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.354+0000 7f8353400640 1 -- 172.21.15.82:0/1997218067 learned_addr learned my addr 172.21.15.82:0/1997218067 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.354+0000 7f8353400640 1 -- 172.21.15.82:0/1997218067 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f835415e150 msgr2=0x7f8354160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.354+0000 7f8353400640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f835415e150 0x7f8354160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.354+0000 7f8352a00640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83540af050 0x7f83541494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.354+0000 7f8353400640 1 -- 172.21.15.82:0/1997218067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83540af050 msgr2=0x7f83541494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.354+0000 7f8353400640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83540af050 0x7f83541494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.354+0000 7f8353400640 1 -- 172.21.15.82:0/1997218067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400094d0 con 0x7f835406dc70 2024-03-16T18:03:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.355+0000 7f8353400640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835406dc70 0x7f8354157570 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f834400c7f0 tx=0x7f834400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.355+0000 7f834b400640 1 -- 172.21.15.82:0/1997218067 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344007820 con 0x7f835406dc70 2024-03-16T18:03:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.355+0000 7f834b400640 1 -- 172.21.15.82:0/1997218067 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8344007980 con 0x7f835406dc70 2024-03-16T18:03:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.355+0000 7f834b400640 1 -- 172.21.15.82:0/1997218067 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8344007d30 con 0x7f835406dc70 2024-03-16T18:03:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.355+0000 7f835a200640 1 -- 172.21.15.82:0/1997218067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835415ff40 con 0x7f835406dc70 2024-03-16T18:03:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.355+0000 7f835a200640 1 -- 172.21.15.82:0/1997218067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8354160500 con 0x7f835406dc70 2024-03-16T18:03:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.356+0000 7f835a200640 1 -- 172.21.15.82:0/1997218067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8354067f30 con 0x7f835406dc70 2024-03-16T18:03:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.357+0000 7f834b400640 1 -- 172.21.15.82:0/1997218067 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f83440113f0 con 0x7f835406dc70 2024-03-16T18:03:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.358+0000 7f834b400640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f833406b3e0 0x7f833406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.358+0000 7f834b400640 1 -- 172.21.15.82:0/1997218067 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8344089ed0 con 0x7f835406dc70 2024-03-16T18:03:47.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.358+0000 7f8352a00640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f833406b3e0 0x7f833406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:47.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.359+0000 7f8352a00640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f833406b3e0 0x7f833406d8a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8340036000 tx=0x7f834003d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.362+0000 7f834b400640 1 -- 172.21.15.82:0/1997218067 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8344054ae0 con 0x7f835406dc70 2024-03-16T18:03:47.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:47.554+0000 7f835a200640 1 -- 172.21.15.82:0/1997218067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f835415d5b0 con 0x7f835406dc70 2024-03-16T18:03:48.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.288+0000 7f834b400640 1 -- 172.21.15.82:0/1997218067 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7f83440581a0 con 0x7f835406dc70 2024-03-16T18:03:48.292 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-03-16T18:03:48.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.292+0000 7f835a200640 1 -- 172.21.15.82:0/1997218067 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f833406b3e0 msgr2=0x7f833406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:48.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.292+0000 7f835a200640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f833406b3e0 0x7f833406d8a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8340036000 tx=0x7f834003d040 comp rx=0 tx=0).stop 2024-03-16T18:03:48.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.292+0000 7f835a200640 1 -- 172.21.15.82:0/1997218067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835406dc70 msgr2=0x7f8354157570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:48.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.292+0000 7f835a200640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835406dc70 0x7f8354157570 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f834400c7f0 tx=0x7f834400ccb0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.293+0000 7f835a200640 1 -- 172.21.15.82:0/1997218067 shutdown_connections 2024-03-16T18:03:48.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.293+0000 7f835a200640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f833406b3e0 0x7f833406d8a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.293+0000 7f835a200640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f835415e150 0x7f8354160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.293+0000 7f835a200640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f83540af050 0x7f83541494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.293+0000 7f835a200640 1 --2- 172.21.15.82:0/1997218067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f835406dc70 0x7f8354157570 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.293+0000 7f835a200640 1 -- 172.21.15.82:0/1997218067 >> 172.21.15.82:0/1997218067 conn(0x7f83540b1740 msgr2=0x7f83540cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:48.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.294+0000 7f835a200640 1 -- 172.21.15.82:0/1997218067 shutdown_connections 2024-03-16T18:03:48.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.294+0000 7f835a200640 1 -- 172.21.15.82:0/1997218067 wait complete. 2024-03-16T18:03:48.320 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.525+0000 7f8b74c00640 1 Processor -- start 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.525+0000 7f8b74c00640 1 -- start start 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.526+0000 7f8b74c00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f8b70159500 0x7f8b7015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.526+0000 7f8b74c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b700ced70 0x7f8b7006c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.526+0000 7f8b74c00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b700cf5a0 con 0x7f8b700ccb80 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.526+0000 7f8b74c00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b700b3770 con 0x7f8b700ced70 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.526+0000 7f8b74c00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b700b3a50 con 0x7f8b70159500 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.527+0000 7f8b6ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b700ced70 0x7f8b7006c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.527+0000 7f8b6ea00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b700ced70 0x7f8b7006c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49134/0 (socket says 172.21.15.82:49134) 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.527+0000 7f8b6ea00640 1 -- 172.21.15.82:0/407869536 learned_addr learned my addr 172.21.15.82:0/407869536 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.527+0000 7f8b6fe00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8b700ccb80 0x7f8b700ccf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56012/0 (socket says 172.21.15.82:56012) 2024-03-16T18:03:48.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.527+0000 7f8b6ea00640 1 -- 172.21.15.82:0/407869536 >> v2:172.21.15.90:3300/0 conn(0x7f8b70159500 msgr2=0x7f8b7015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:48.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.527+0000 7f8b6ea00640 1 --2- 172.21.15.82:0/407869536 >> v2:172.21.15.90:3300/0 conn(0x7f8b70159500 0x7f8b7015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.527+0000 7f8b6ea00640 1 -- 172.21.15.82:0/407869536 >> v1:172.21.15.82:6789/0 conn(0x7f8b700ccb80 legacy=0x7f8b700ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:48.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.527+0000 7f8b6ea00640 1 -- 172.21.15.82:0/407869536 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b700b3d30 con 0x7f8b700ced70 2024-03-16T18:03:48.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.527+0000 7f8b6e000640 1 -- 172.21.15.82:0/407869536 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979036195 0 0) 0x7f8b700cf5a0 con 0x7f8b700ccb80 2024-03-16T18:03:48.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.529+0000 7f8b6ea00640 1 --2- 172.21.15.82:0/407869536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 0x7f8b7006c8b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8b700cf860 tx=0x7f8b5c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f476b454689b2ad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:48.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.529+0000 7f8b6e000640 1 -- 172.21.15.82:0/407869536 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c030860 con 0x7f8b700ced70 2024-03-16T18:03:48.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.529+0000 7f8b6e000640 1 -- 172.21.15.82:0/407869536 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c0309c0 con 0x7f8b700ced70 2024-03-16T18:03:48.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.529+0000 7f8b6e000640 1 -- 172.21.15.82:0/407869536 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c030d50 con 0x7f8b700ced70 2024-03-16T18:03:48.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.530+0000 7f8b74c00640 1 -- 172.21.15.82:0/407869536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 msgr2=0x7f8b7006c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.530+0000 7f8b74c00640 1 --2- 172.21.15.82:0/407869536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 0x7f8b7006c8b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8b700cf860 tx=0x7f8b5c02b9a0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.530+0000 7f8b74c00640 1 -- 172.21.15.82:0/407869536 shutdown_connections 2024-03-16T18:03:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.530+0000 7f8b74c00640 1 --2- 172.21.15.82:0/407869536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 0x7f8b7006c8b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.530+0000 7f8b74c00640 1 --2- 172.21.15.82:0/407869536 >> v2:172.21.15.90:3300/0 conn(0x7f8b70159500 0x7f8b7015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.530+0000 7f8b74c00640 1 -- 172.21.15.82:0/407869536 >> 172.21.15.82:0/407869536 conn(0x7f8b7006b050 msgr2=0x7f8b7006b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.530+0000 7f8b74c00640 1 -- 172.21.15.82:0/407869536 shutdown_connections 2024-03-16T18:03:48.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.530+0000 7f8b74c00640 1 -- 172.21.15.82:0/407869536 wait complete. 2024-03-16T18:03:48.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.531+0000 7f8b74c00640 1 Processor -- start 2024-03-16T18:03:48.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.532+0000 7f8b74c00640 1 -- start start 2024-03-16T18:03:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.532+0000 7f8b74c00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b700ccb80 0x7f8b70156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.532+0000 7f8b74c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 0x7f8b700c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.532+0000 7f8b74c00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8b70159500 0x7f8b700b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.532+0000 7f8b74c00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b70070870 con 0x7f8b700ced70 2024-03-16T18:03:48.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.532+0000 7f8b74c00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b70070b10 con 0x7f8b700ccb80 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.532+0000 7f8b74c00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b70070df0 con 0x7f8b70159500 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.533+0000 7f8b6ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 0x7f8b700c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.533+0000 7f8b6ea00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 0x7f8b700c4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49144/0 (socket says 172.21.15.82:49144) 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.533+0000 7f8b6ea00640 1 -- 172.21.15.82:0/2696095098 learned_addr learned my addr 172.21.15.82:0/2696095098 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.533+0000 7f8b6f400640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b700ccb80 0x7f8b70156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.533+0000 7f8b6ea00640 1 -- 172.21.15.82:0/2696095098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8b70159500 msgr2=0x7f8b700b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.533+0000 7f8b6fe00640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8b70159500 0x7f8b700b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.533+0000 7f8b6ea00640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8b70159500 0x7f8b700b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.533+0000 7f8b6ea00640 1 -- 172.21.15.82:0/2696095098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b700ccb80 msgr2=0x7f8b70156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.533+0000 7f8b6ea00640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b700ccb80 0x7f8b70156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.533+0000 7f8b6ea00640 1 -- 172.21.15.82:0/2696095098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c0094d0 con 0x7f8b700ced70 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.534+0000 7f8b6ea00640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 0x7f8b700c4920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0090b0 tx=0x7f8b5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:48.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.534+0000 7f8b67400640 1 -- 172.21.15.82:0/2696095098 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c03e040 con 0x7f8b700ced70 2024-03-16T18:03:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.534+0000 7f8b67400640 1 -- 172.21.15.82:0/2696095098 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c039d50 con 0x7f8b700ced70 2024-03-16T18:03:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.534+0000 7f8b74c00640 1 -- 172.21.15.82:0/2696095098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b70159a60 con 0x7f8b700ced70 2024-03-16T18:03:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.534+0000 7f8b67400640 1 -- 172.21.15.82:0/2696095098 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c004270 con 0x7f8b700ced70 2024-03-16T18:03:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.534+0000 7f8b74c00640 1 -- 172.21.15.82:0/2696095098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b700713b0 con 0x7f8b700ced70 2024-03-16T18:03:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.535+0000 7f8b74c00640 1 -- 172.21.15.82:0/2696095098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7004f250 con 0x7f8b700ced70 2024-03-16T18:03:48.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.536+0000 7f8b67400640 1 -- 172.21.15.82:0/2696095098 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8b5c0043d0 con 0x7f8b700ced70 2024-03-16T18:03:48.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.537+0000 7f8b67400640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b5006b3b0 0x7f8b5006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:48.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.537+0000 7f8b67400640 1 -- 172.21.15.82:0/2696095098 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f8b5c0ac3e0 con 0x7f8b700ced70 2024-03-16T18:03:48.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.537+0000 7f8b6f400640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b5006b3b0 0x7f8b5006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:48.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.538+0000 7f8b6f400640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b5006b3b0 0x7f8b5006d870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000a770 tx=0x7f8b60008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:48.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.542+0000 7f8b67400640 1 -- 172.21.15.82:0/2696095098 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8b5c0771a0 con 0x7f8b700ced70 2024-03-16T18:03:48.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:48.740+0000 7f8b74c00640 1 -- 172.21.15.82:0/2696095098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b70156540 con 0x7f8b700ced70 2024-03-16T18:03:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.292+0000 7f8b67400640 1 -- 172.21.15.82:0/2696095098 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8b5c07a860 con 0x7f8b700ced70 2024-03-16T18:03:49.296 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-03-16T18:03:49.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.296+0000 7f8b74c00640 1 -- 172.21.15.82:0/2696095098 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b5006b3b0 msgr2=0x7f8b5006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.296+0000 7f8b74c00640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b5006b3b0 0x7f8b5006d870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b6000a770 tx=0x7f8b60008040 comp rx=0 tx=0).stop 2024-03-16T18:03:49.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.296+0000 7f8b74c00640 1 -- 172.21.15.82:0/2696095098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 msgr2=0x7f8b700c4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.296+0000 7f8b74c00640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 0x7f8b700c4920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0090b0 tx=0x7f8b5c004730 comp rx=0 tx=0).stop 2024-03-16T18:03:49.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.297+0000 7f8b74c00640 1 -- 172.21.15.82:0/2696095098 shutdown_connections 2024-03-16T18:03:49.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.297+0000 7f8b74c00640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b5006b3b0 0x7f8b5006d870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.297+0000 7f8b74c00640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8b70159500 0x7f8b700b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.297+0000 7f8b74c00640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b700ced70 0x7f8b700c4920 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.297+0000 7f8b74c00640 1 --2- 172.21.15.82:0/2696095098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b700ccb80 0x7f8b70156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.297+0000 7f8b74c00640 1 -- 172.21.15.82:0/2696095098 >> 172.21.15.82:0/2696095098 conn(0x7f8b7006b050 msgr2=0x7f8b700b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:49.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.299+0000 7f8b74c00640 1 -- 172.21.15.82:0/2696095098 shutdown_connections 2024-03-16T18:03:49.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.299+0000 7f8b74c00640 1 -- 172.21.15.82:0/2696095098 wait complete. 2024-03-16T18:03:49.330 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-16T18:03:49.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.523+0000 7f3818a00640 1 Processor -- start 2024-03-16T18:03:49.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.523+0000 7f3818a00640 1 -- start start 2024-03-16T18:03:49.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.524+0000 7f3818a00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f3814159960 0x7f381406e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.524+0000 7f3818a00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f381406e770 0x7f38140cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.524+0000 7f3818a00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38140ce550 con 0x7f38140cc850 2024-03-16T18:03:49.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.524+0000 7f3818a00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3814159d40 con 0x7f381406e770 2024-03-16T18:03:49.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.524+0000 7f3818a00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381406ec20 con 0x7f3814159960 2024-03-16T18:03:49.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.525+0000 7f3809400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f381406e770 0x7f38140cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.525+0000 7f3809400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f381406e770 0x7f38140cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49154/0 (socket says 172.21.15.82:49154) 2024-03-16T18:03:49.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.525+0000 7f3809400640 1 -- 172.21.15.82:0/3343840317 learned_addr learned my addr 172.21.15.82:0/3343840317 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:49.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.525+0000 7f3811e00640 1 --2- 172.21.15.82:0/3343840317 >> v2:172.21.15.90:3300/0 conn(0x7f3814159960 0x7f381406e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.525+0000 7f3809400640 1 -- 172.21.15.82:0/3343840317 >> v2:172.21.15.90:3300/0 conn(0x7f3814159960 msgr2=0x7f381406e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:49.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.525+0000 7f3809400640 1 --2- 172.21.15.82:0/3343840317 >> v2:172.21.15.90:3300/0 conn(0x7f3814159960 0x7f381406e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.525+0000 7f3809400640 1 -- 172.21.15.82:0/3343840317 >> v1:172.21.15.90:6789/0 conn(0x7f38140cc850 legacy=0x7f381415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:49.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.525+0000 7f3809400640 1 -- 172.21.15.82:0/3343840317 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381415c370 con 0x7f381406e770 2024-03-16T18:03:49.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.525+0000 7f3811e00640 1 --2- 172.21.15.82:0/3343840317 >> v2:172.21.15.90:3300/0 conn(0x7f3814159960 0x7f381406e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:49.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.526+0000 7f3809400640 1 --2- 172.21.15.82:0/3343840317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 0x7f38140cc310 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38140cc260 tx=0x7f37fc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e3ffaf77c0223d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:49.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.527+0000 7f3811400640 1 -- 172.21.15.82:0/3343840317 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0317e0 con 0x7f381406e770 2024-03-16T18:03:49.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.527+0000 7f3811400640 1 -- 172.21.15.82:0/3343840317 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc031940 con 0x7f381406e770 2024-03-16T18:03:49.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.527+0000 7f3811400640 1 -- 172.21.15.82:0/3343840317 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc031cd0 con 0x7f381406e770 2024-03-16T18:03:49.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.527+0000 7f3818a00640 1 -- 172.21.15.82:0/3343840317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 msgr2=0x7f38140cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.527+0000 7f3818a00640 1 --2- 172.21.15.82:0/3343840317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 0x7f38140cc310 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38140cc260 tx=0x7f37fc02b880 comp rx=0 tx=0).stop 2024-03-16T18:03:49.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.528+0000 7f3818a00640 1 -- 172.21.15.82:0/3343840317 shutdown_connections 2024-03-16T18:03:49.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.528+0000 7f3818a00640 1 --2- 172.21.15.82:0/3343840317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 0x7f38140cc310 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.528+0000 7f3818a00640 1 --2- 172.21.15.82:0/3343840317 >> v2:172.21.15.90:3300/0 conn(0x7f3814159960 0x7f381406e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.528+0000 7f3818a00640 1 -- 172.21.15.82:0/3343840317 >> 172.21.15.82:0/3343840317 conn(0x7f38140b1640 msgr2=0x7f38140b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:49.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.528+0000 7f3818a00640 1 -- 172.21.15.82:0/3343840317 shutdown_connections 2024-03-16T18:03:49.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.528+0000 7f3818a00640 1 -- 172.21.15.82:0/3343840317 wait complete. 2024-03-16T18:03:49.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.529+0000 7f3818a00640 1 Processor -- start 2024-03-16T18:03:49.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.529+0000 7f3818a00640 1 -- start start 2024-03-16T18:03:49.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.530+0000 7f3818a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 0x7f38141585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.530+0000 7f3818a00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f38140cc850 0x7f3814140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.530+0000 7f3818a00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3814159960 0x7f381415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.530+0000 7f3818a00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381415cff0 con 0x7f381406e770 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.530+0000 7f3818a00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381413ac80 con 0x7f3814159960 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.530+0000 7f3818a00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381413eca0 con 0x7f38140cc850 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3811e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 0x7f38141585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3811e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 0x7f38141585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49170/0 (socket says 172.21.15.82:49170) 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3811e00640 1 -- 172.21.15.82:0/2208178369 learned_addr learned my addr 172.21.15.82:0/2208178369 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3812800640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3814159960 0x7f381415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3809400640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f38140cc850 0x7f3814140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3811e00640 1 -- 172.21.15.82:0/2208178369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f38140cc850 msgr2=0x7f3814140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3811e00640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f38140cc850 0x7f3814140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3811e00640 1 -- 172.21.15.82:0/2208178369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3814159960 msgr2=0x7f381415e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3811e00640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3814159960 0x7f381415e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3811e00640 1 -- 172.21.15.82:0/2208178369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc0094d0 con 0x7f381406e770 2024-03-16T18:03:49.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3812800640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3814159960 0x7f381415e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:49.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.531+0000 7f3811e00640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 0x7f38141585e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f380400c7f0 tx=0x7f380400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:49.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.532+0000 7f380aa00640 1 -- 172.21.15.82:0/2208178369 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804007820 con 0x7f381406e770 2024-03-16T18:03:49.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.532+0000 7f380aa00640 1 -- 172.21.15.82:0/2208178369 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3804007980 con 0x7f381406e770 2024-03-16T18:03:49.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.532+0000 7f380aa00640 1 -- 172.21.15.82:0/2208178369 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804007cf0 con 0x7f381406e770 2024-03-16T18:03:49.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.532+0000 7f3818a00640 1 -- 172.21.15.82:0/2208178369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381413ef80 con 0x7f381406e770 2024-03-16T18:03:49.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.532+0000 7f3818a00640 1 -- 172.21.15.82:0/2208178369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381413f540 con 0x7f381406e770 2024-03-16T18:03:49.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.533+0000 7f3818a00640 1 -- 172.21.15.82:0/2208178369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3814067f30 con 0x7f381406e770 2024-03-16T18:03:49.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.534+0000 7f380aa00640 1 -- 172.21.15.82:0/2208178369 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f38040113f0 con 0x7f381406e770 2024-03-16T18:03:49.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.535+0000 7f380aa00640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f37f006b3b0 0x7f37f006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.535+0000 7f380aa00640 1 -- 172.21.15.82:0/2208178369 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3804089c90 con 0x7f381406e770 2024-03-16T18:03:49.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.536+0000 7f3809400640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f37f006b3b0 0x7f37f006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.536+0000 7f3809400640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f37f006b3b0 0x7f37f006d870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37fc036000 tx=0x7f37fc03d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:49.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.540+0000 7f380aa00640 1 -- 172.21.15.82:0/2208178369 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3804054bf0 con 0x7f381406e770 2024-03-16T18:03:49.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.738+0000 7f3818a00640 1 -- 172.21.15.82:0/2208178369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f38140ceb80 con 0x7f381406e770 2024-03-16T18:03:49.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.745+0000 7f380aa00640 1 -- 172.21.15.82:0/2208178369 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f38040582b0 con 0x7f381406e770 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.747+0000 7f3818a00640 1 -- 172.21.15.82:0/2208178369 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f37f006b3b0 msgr2=0x7f37f006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.747+0000 7f3818a00640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f37f006b3b0 0x7f37f006d870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f37fc036000 tx=0x7f37fc03d040 comp rx=0 tx=0).stop 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.748+0000 7f3818a00640 1 -- 172.21.15.82:0/2208178369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 msgr2=0x7f38141585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.748+0000 7f3818a00640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 0x7f38141585e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f380400c7f0 tx=0x7f380400ccb0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.748+0000 7f3818a00640 1 -- 172.21.15.82:0/2208178369 shutdown_connections 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.748+0000 7f3818a00640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f37f006b3b0 0x7f37f006d870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.748+0000 7f3818a00640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3814159960 0x7f381415e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.748+0000 7f3818a00640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f38140cc850 0x7f3814140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.748+0000 7f3818a00640 1 --2- 172.21.15.82:0/2208178369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f381406e770 0x7f38141585e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.748+0000 7f3818a00640 1 -- 172.21.15.82:0/2208178369 >> 172.21.15.82:0/2208178369 conn(0x7f38140b1640 msgr2=0x7f3814144ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.748+0000 7f3818a00640 1 -- 172.21.15.82:0/2208178369 shutdown_connections 2024-03-16T18:03:49.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.748+0000 7f3818a00640 1 -- 172.21.15.82:0/2208178369 wait complete. 2024-03-16T18:03:49.772 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-03-16T18:03:49.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.972+0000 7f66ba800640 1 Processor -- start 2024-03-16T18:03:49.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.972+0000 7f66ba800640 1 -- start start 2024-03-16T18:03:49.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.974+0000 7f66ba800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66b41454d0 0x7f66b40b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.974+0000 7f66ba800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f66b40d0040 0x7f66b4070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.974+0000 7f66ba800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b40b4b90 con 0x7f66b40cdf80 2024-03-16T18:03:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.974+0000 7f66ba800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b40cf500 con 0x7f66b40d0040 2024-03-16T18:03:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.974+0000 7f66ba800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b41458b0 con 0x7f66b41454d0 2024-03-16T18:03:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.974+0000 7f66b3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f66b40d0040 0x7f66b4070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.974+0000 7f66b3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f66b40d0040 0x7f66b4070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49178/0 (socket says 172.21.15.82:49178) 2024-03-16T18:03:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.974+0000 7f66b3e00640 1 -- 172.21.15.82:0/1648259121 learned_addr learned my addr 172.21.15.82:0/1648259121 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.974+0000 7f66b3400640 1 --2- 172.21.15.82:0/1648259121 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66b41454d0 0x7f66b40b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.975+0000 7f66b3e00640 1 -- 172.21.15.82:0/1648259121 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66b41454d0 msgr2=0x7f66b40b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.975+0000 7f66b3e00640 1 --2- 172.21.15.82:0/1648259121 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66b41454d0 0x7f66b40b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.975+0000 7f66b3e00640 1 -- 172.21.15.82:0/1648259121 >> v1:172.21.15.90:6789/0 conn(0x7f66b40cdf80 legacy=0x7f66b40cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.975+0000 7f66b3e00640 1 -- 172.21.15.82:0/1648259121 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40ce460 con 0x7f66b40d0040 2024-03-16T18:03:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.976+0000 7f66b3e00640 1 --2- 172.21.15.82:0/1648259121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 0x7f66b4070980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009e50 tx=0x7f66ac02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd22202ba26c6c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:49.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.976+0000 7f66b2000640 1 -- 172.21.15.82:0/1648259121 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac02cb00 con 0x7f66b40d0040 2024-03-16T18:03:49.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.976+0000 7f66b2000640 1 -- 172.21.15.82:0/1648259121 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac02cc60 con 0x7f66b40d0040 2024-03-16T18:03:49.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.976+0000 7f66b2000640 1 -- 172.21.15.82:0/1648259121 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0343f0 con 0x7f66b40d0040 2024-03-16T18:03:49.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.977+0000 7f66ba800640 1 -- 172.21.15.82:0/1648259121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 msgr2=0x7f66b4070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.977+0000 7f66ba800640 1 --2- 172.21.15.82:0/1648259121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 0x7f66b4070980 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009e50 tx=0x7f66ac02bf60 comp rx=0 tx=0).stop 2024-03-16T18:03:49.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.977+0000 7f66ba800640 1 -- 172.21.15.82:0/1648259121 shutdown_connections 2024-03-16T18:03:49.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.977+0000 7f66ba800640 1 --2- 172.21.15.82:0/1648259121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 0x7f66b4070980 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.977+0000 7f66ba800640 1 --2- 172.21.15.82:0/1648259121 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66b41454d0 0x7f66b40b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.977+0000 7f66ba800640 1 -- 172.21.15.82:0/1648259121 >> 172.21.15.82:0/1648259121 conn(0x7f66b406ae70 msgr2=0x7f66b406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:49.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.977+0000 7f66ba800640 1 -- 172.21.15.82:0/1648259121 shutdown_connections 2024-03-16T18:03:49.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.977+0000 7f66ba800640 1 -- 172.21.15.82:0/1648259121 wait complete. 2024-03-16T18:03:49.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.979+0000 7f66ba800640 1 Processor -- start 2024-03-16T18:03:49.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.979+0000 7f66ba800640 1 -- start start 2024-03-16T18:03:49.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66ba800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66b40cdf80 0x7f66b413d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66ba800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 0x7f66b40af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66ba800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66b41454d0 0x7f66b4157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66ba800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40b1260 con 0x7f66b40d0040 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66ba800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40afe10 con 0x7f66b41454d0 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66ba800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b413dbc0 con 0x7f66b40cdf80 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66b2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 0x7f66b40af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66b2a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 0x7f66b40af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49180/0 (socket says 172.21.15.82:49180) 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66b2a00640 1 -- 172.21.15.82:0/1445034479 learned_addr learned my addr 172.21.15.82:0/1445034479 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66b3400640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66b40cdf80 0x7f66b413d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66b3e00640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66b41454d0 0x7f66b4157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66b2a00640 1 -- 172.21.15.82:0/1445034479 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66b40cdf80 msgr2=0x7f66b413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66b2a00640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66b40cdf80 0x7f66b413d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66b2a00640 1 -- 172.21.15.82:0/1445034479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66b41454d0 msgr2=0x7f66b4157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:49.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.980+0000 7f66b2a00640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66b41454d0 0x7f66b4157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:49.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.981+0000 7f66b2a00640 1 -- 172.21.15.82:0/1445034479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac009b30 con 0x7f66b40d0040 2024-03-16T18:03:49.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.981+0000 7f66b2a00640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 0x7f66b40af8d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c430 tx=0x7f66a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:49.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.981+0000 7f66ab400640 1 -- 172.21.15.82:0/1445034479 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a00052d0 con 0x7f66b40d0040 2024-03-16T18:03:49.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.981+0000 7f66ab400640 1 -- 172.21.15.82:0/1445034479 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a0005450 con 0x7f66b40d0040 2024-03-16T18:03:49.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.981+0000 7f66ab400640 1 -- 172.21.15.82:0/1445034479 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0007260 con 0x7f66b40d0040 2024-03-16T18:03:49.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.981+0000 7f66ba800640 1 -- 172.21.15.82:0/1445034479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b413dea0 con 0x7f66b40d0040 2024-03-16T18:03:49.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.981+0000 7f66ba800640 1 -- 172.21.15.82:0/1445034479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b413e460 con 0x7f66b40d0040 2024-03-16T18:03:49.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.982+0000 7f66ba800640 1 -- 172.21.15.82:0/1445034479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b4067f30 con 0x7f66b40d0040 2024-03-16T18:03:49.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.983+0000 7f66ab400640 1 -- 172.21.15.82:0/1445034479 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f66a00073c0 con 0x7f66b40d0040 2024-03-16T18:03:49.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.985+0000 7f66ab400640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f669406b3b0 0x7f669406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:49.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.985+0000 7f66ab400640 1 -- 172.21.15.82:0/1445034479 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f66a0088d60 con 0x7f66b40d0040 2024-03-16T18:03:49.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.985+0000 7f66b3400640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f669406b3b0 0x7f669406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:49.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.986+0000 7f66b3400640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f669406b3b0 0x7f669406d870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f66a4006fd0 tx=0x7f66a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:49.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:49.989+0000 7f66ab400640 1 -- 172.21.15.82:0/1445034479 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f66a0053cb0 con 0x7f66b40d0040 2024-03-16T18:03:50.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.255+0000 7f66ba800640 1 -- 172.21.15.82:0/1445034479 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) v1 -- 0x7f66b40719e0 con 0x7f66b40d0040 2024-03-16T18:03:50.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.308+0000 7f66ab400640 1 -- 172.21.15.82:0/1445034479 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v244) v1 ==== 120+0+0 (secure 0 0 0) 0x7f66a0057370 con 0x7f66b40d0040 2024-03-16T18:03:50.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 -- 172.21.15.82:0/1445034479 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f669406b3b0 msgr2=0x7f669406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:50.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f669406b3b0 0x7f669406d870 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f66a4006fd0 tx=0x7f66a4008040 comp rx=0 tx=0).stop 2024-03-16T18:03:50.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 -- 172.21.15.82:0/1445034479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 msgr2=0x7f66b40af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:50.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 0x7f66b40af8d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c430 tx=0x7f66a000c8f0 comp rx=0 tx=0).stop 2024-03-16T18:03:50.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 -- 172.21.15.82:0/1445034479 shutdown_connections 2024-03-16T18:03:50.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f669406b3b0 0x7f669406d870 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:50.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66b41454d0 0x7f66b4157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:50.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66b40d0040 0x7f66b40af8d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:50.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 --2- 172.21.15.82:0/1445034479 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f66b40cdf80 0x7f66b413d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:50.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 -- 172.21.15.82:0/1445034479 >> 172.21.15.82:0/1445034479 conn(0x7f66b406ae70 msgr2=0x7f66b406c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:50.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.311+0000 7f66ba800640 1 -- 172.21.15.82:0/1445034479 shutdown_connections 2024-03-16T18:03:50.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:50.312+0000 7f66ba800640 1 -- 172.21.15.82:0/1445034479 wait complete. 2024-03-16T18:03:50.335 INFO:tasks.cephfs_test_runner:test_acls (tasks.cephfs.test_acls.TestACLs) ... ok 2024-03-16T18:03:50.336 INFO:tasks.cephfs_test_runner: 2024-03-16T18:03:50.336 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-16T18:03:50.336 INFO:tasks.cephfs_test_runner:Ran 1 test in 331.065s 2024-03-16T18:03:50.336 INFO:tasks.cephfs_test_runner: 2024-03-16T18:03:50.336 INFO:tasks.cephfs_test_runner:OK 2024-03-16T18:03:50.336 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-16T18:03:50.349 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-16T18:03:50.368 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-16T18:03:50.368 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T18:03:50.368 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T18:03:50.422 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-16T18:03:50.422 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-03-16T18:03:50.422 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T18:03:50.518 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053412,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-16T18:03:50.518 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T18:03:50.518 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T18:03:50.575 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-16T18:03:50.576 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-03-16T18:03:50.576 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T18:03:50.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053413,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249168k,nr_inodes=812292,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498340k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-16T18:03:50.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:50.626 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-16T18:03:50.682 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-16T18:03:50.683 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-16T18:03:50.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:50.683 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T18:03:50.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:50.779 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-16T18:03:50.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:50.875 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-03-16T18:03:50.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:50.999 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-03-16T18:03:51.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:51.045 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-03-16T18:03:51.175 INFO:teuthology.orchestra.run.smithi082.stderr:Cannot find device "ceph-brx" 2024-03-16T18:03:51.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T18:03:51.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:51.180 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-03-16T18:03:51.237 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-16T18:03:51.237 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-16T18:03:51.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:51.238 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-16T18:03:51.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:51.337 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-16T18:03:51.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:51.431 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-03-16T18:03:51.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:51.541 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-03-16T18:03:51.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T18:03:51.586 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-03-16T18:03:51.728 INFO:teuthology.orchestra.run.smithi090.stderr:Cannot find device "ceph-brx" 2024-03-16T18:03:51.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T18:03:51.732 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-16T18:03:51.743 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-16T18:03:51.743 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T18:03:51.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.959+0000 7f162f200640 1 Processor -- start 2024-03-16T18:03:51.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.959+0000 7f162f200640 1 -- start start 2024-03-16T18:03:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.960+0000 7f162f200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f16280aee40 0x7f162806d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.961+0000 7f162f200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16280b3370 con 0x7f16281595b0 2024-03-16T18:03:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.961+0000 7f162f200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162806dd20 con 0x7f1628159990 2024-03-16T18:03:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.961+0000 7f162f200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162815d1c0 con 0x7f16280aee40 2024-03-16T18:03:51.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.961+0000 7f162de00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f16281595b0 0x7f16280ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56022/0 (socket says 172.21.15.82:56022) 2024-03-16T18:03:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.961+0000 7f162de00640 1 -- 172.21.15.82:0/92666100 learned_addr learned my addr 172.21.15.82:0/92666100 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.961+0000 7f162d400640 1 --2- 172.21.15.82:0/92666100 >> v2:172.21.15.82:3300/0 conn(0x7f16280aee40 0x7f162806d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.961+0000 7f162d400640 1 -- 172.21.15.82:0/92666100 >> v1:172.21.15.90:6789/0 conn(0x7f1628159990 legacy=0x7f162806d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.961+0000 7f162d400640 1 -- 172.21.15.82:0/92666100 >> v1:172.21.15.82:6789/0 conn(0x7f16281595b0 legacy=0x7f16280ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.961+0000 7f162ca00640 1 -- 172.21.15.82:0/92666100 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569830287 0 0) 0x7f16280b3370 con 0x7f16281595b0 2024-03-16T18:03:51.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.961+0000 7f162d400640 1 -- 172.21.15.82:0/92666100 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162815f7a0 con 0x7f16280aee40 2024-03-16T18:03:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.962+0000 7f162d400640 1 --2- 172.21.15.82:0/92666100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 0x7f162806d030 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f16180097a0 tx=0x7f161802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b16065bb99f5871 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.963+0000 7f162ca00640 1 -- 172.21.15.82:0/92666100 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161802e070 con 0x7f16280aee40 2024-03-16T18:03:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.963+0000 7f162ca00640 1 -- 172.21.15.82:0/92666100 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16180315a0 con 0x7f16280aee40 2024-03-16T18:03:51.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.963+0000 7f162ca00640 1 -- 172.21.15.82:0/92666100 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618031930 con 0x7f16280aee40 2024-03-16T18:03:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.963+0000 7f162f200640 1 -- 172.21.15.82:0/92666100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 msgr2=0x7f162806d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.963+0000 7f162f200640 1 --2- 172.21.15.82:0/92666100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 0x7f162806d030 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f16180097a0 tx=0x7f161802b840 comp rx=0 tx=0).stop 2024-03-16T18:03:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.963+0000 7f162f200640 1 -- 172.21.15.82:0/92666100 shutdown_connections 2024-03-16T18:03:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.963+0000 7f162f200640 1 --2- 172.21.15.82:0/92666100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 0x7f162806d030 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.963+0000 7f162f200640 1 -- 172.21.15.82:0/92666100 >> 172.21.15.82:0/92666100 conn(0x7f162806ae70 msgr2=0x7f162806b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:51.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.963+0000 7f162f200640 1 -- 172.21.15.82:0/92666100 shutdown_connections 2024-03-16T18:03:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.964+0000 7f162f200640 1 -- 172.21.15.82:0/92666100 wait complete. 2024-03-16T18:03:51.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.965+0000 7f162f200640 1 Processor -- start 2024-03-16T18:03:51.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.965+0000 7f162f200640 1 -- start start 2024-03-16T18:03:51.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.966+0000 7f162f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 0x7f1628144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:51.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.966+0000 7f162f200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f16281595b0 0x7f1628157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.966+0000 7f162f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1628159990 0x7f16280ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.966+0000 7f162f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1628158c80 con 0x7f16280aee40 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.966+0000 7f162f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1628158070 con 0x7f1628159990 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.966+0000 7f162f200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1628158350 con 0x7f16281595b0 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.966+0000 7f162de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 0x7f1628144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.967+0000 7f162de00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 0x7f1628144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49210/0 (socket says 172.21.15.82:49210) 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.967+0000 7f162de00640 1 -- 172.21.15.82:0/4084849752 learned_addr learned my addr 172.21.15.82:0/4084849752 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.967+0000 7f162de00640 1 -- 172.21.15.82:0/4084849752 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f16281595b0 msgr2=0x7f1628157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.967+0000 7f162de00640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f16281595b0 0x7f1628157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.967+0000 7f162de00640 1 -- 172.21.15.82:0/4084849752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1628159990 msgr2=0x7f16280ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.967+0000 7f162e800640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1628159990 0x7f16280ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.967+0000 7f162de00640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1628159990 0x7f16280ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.967+0000 7f162de00640 1 -- 172.21.15.82:0/4084849752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1618009400 con 0x7f16280aee40 2024-03-16T18:03:51.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.967+0000 7f162de00640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 0x7f1628144030 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f161c00c4d0 tx=0x7f161c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.967+0000 7f1622000640 1 -- 172.21.15.82:0/4084849752 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c0074e0 con 0x7f16280aee40 2024-03-16T18:03:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.968+0000 7f1622000640 1 -- 172.21.15.82:0/4084849752 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c007640 con 0x7f16280aee40 2024-03-16T18:03:51.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.968+0000 7f1622000640 1 -- 172.21.15.82:0/4084849752 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c004a20 con 0x7f16280aee40 2024-03-16T18:03:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.968+0000 7f162f200640 1 -- 172.21.15.82:0/4084849752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1628158630 con 0x7f16280aee40 2024-03-16T18:03:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.968+0000 7f162f200640 1 -- 172.21.15.82:0/4084849752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1628144570 con 0x7f16280aee40 2024-03-16T18:03:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.969+0000 7f162f200640 1 -- 172.21.15.82:0/4084849752 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162804f250 con 0x7f16280aee40 2024-03-16T18:03:51.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.970+0000 7f1622000640 1 -- 172.21.15.82:0/4084849752 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f161c004b80 con 0x7f16280aee40 2024-03-16T18:03:51.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.971+0000 7f1622000640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f160c06b3b0 0x7f160c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:51.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.971+0000 7f1622000640 1 -- 172.21.15.82:0/4084849752 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f161c088f70 con 0x7f16280aee40 2024-03-16T18:03:51.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.971+0000 7f162d400640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f160c06b3b0 0x7f160c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:51.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.972+0000 7f162d400640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f160c06b3b0 0x7f160c06d870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1618036000 tx=0x7f1618030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:51.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:51.976+0000 7f1622000640 1 -- 172.21.15.82:0/4084849752 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f161c053e30 con 0x7f16280aee40 2024-03-16T18:03:52.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.169+0000 7f162f200640 1 -- 172.21.15.82:0/4084849752 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1628160040 con 0x7f160c06b3b0 2024-03-16T18:03:52.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.171+0000 7f1622000640 1 -- 172.21.15.82:0/4084849752 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13496 (secure 0 0 0) 0x7f1618031a70 con 0x7f160c06b3b0 2024-03-16T18:03:52.175 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:52.175 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T18:03:52.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.173+0000 7f162f200640 1 -- 172.21.15.82:0/4084849752 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f160c06b3b0 msgr2=0x7f160c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.173+0000 7f162f200640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f160c06b3b0 0x7f160c06d870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1618036000 tx=0x7f1618030000 comp rx=0 tx=0).stop 2024-03-16T18:03:52.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.173+0000 7f162f200640 1 -- 172.21.15.82:0/4084849752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 msgr2=0x7f1628144030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.173+0000 7f162f200640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 0x7f1628144030 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f161c00c4d0 tx=0x7f161c00c990 comp rx=0 tx=0).stop 2024-03-16T18:03:52.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.173+0000 7f162f200640 1 -- 172.21.15.82:0/4084849752 shutdown_connections 2024-03-16T18:03:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.173+0000 7f162f200640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f160c06b3b0 0x7f160c06d870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.173+0000 7f162f200640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1628159990 0x7f16280ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.173+0000 7f162f200640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f16281595b0 0x7f1628157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.173+0000 7f162f200640 1 --2- 172.21.15.82:0/4084849752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16280aee40 0x7f1628144030 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.173+0000 7f162f200640 1 -- 172.21.15.82:0/4084849752 >> 172.21.15.82:0/4084849752 conn(0x7f162806ae70 msgr2=0x7f162806cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.174+0000 7f162f200640 1 -- 172.21.15.82:0/4084849752 shutdown_connections 2024-03-16T18:03:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.174+0000 7f162f200640 1 -- 172.21.15.82:0/4084849752 wait complete. 2024-03-16T18:03:52.197 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-03-16T18:03:52.123515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221976,"kb_used_data":5816,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752744,"statfs":{"total":773094113280,"available":772866809856,"internally_reserved":0,"allocated":5955584,"data_stored":3990528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-780544,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-5,"num_write":-157,"num_write_kb":-819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.024061"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-16T18:03:46.108312+0000","last_change":"2024-03-16T17:57:35.560172+0000","last_active":"2024-03-16T18:03:46.108312+0000","last_peered":"2024-03-16T18:03:46.108312+0000","last_clean":"2024-03-16T18:03:46.108312+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T18:03:46.108312+0000","last_undegraded":"2024-03-16T18:03:46.108312+0000","last_fullsized":"2024-03-16T18:03:46.108312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_clean_scrub_stamp":"2024-03-16T17:57:34.174437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:20:03.039404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542223,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":746546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542223,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":156704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542221,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":161709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542221,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":161141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542222,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":930907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542221,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":156158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":930907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":987136,"data_stored":746456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T18:03:52.198 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T18:03:52.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.409+0000 7fe56ee00640 1 Processor -- start 2024-03-16T18:03:52.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.410+0000 7fe56ee00640 1 -- start start 2024-03-16T18:03:52.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.411+0000 7fe56ee00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe56815dee0 0x7fe5680b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.411+0000 7fe56ee00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe56815e2c0 0x7fe56806d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.411+0000 7fe56ee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56806d8c0 con 0x7fe568072310 2024-03-16T18:03:52.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.411+0000 7fe56ee00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56815e790 con 0x7fe56815dee0 2024-03-16T18:03:52.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.411+0000 7fe56ee00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56815ea70 con 0x7fe56815e2c0 2024-03-16T18:03:52.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe56ce00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe568072310 0x7fe5680ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56024/0 (socket says 172.21.15.82:56024) 2024-03-16T18:03:52.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe56ce00640 1 -- 172.21.15.82:0/51497341 learned_addr learned my addr 172.21.15.82:0/51497341 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:52.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe567e00640 1 --2- 172.21.15.82:0/51497341 >> v2:172.21.15.90:3300/0 conn(0x7fe56815dee0 0x7fe5680b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe55f400640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe56815e2c0 0x7fe56806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639475158 0 0) 0x7fe560001110 con 0x7fe568072310 2024-03-16T18:03:52.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe54c003520 con 0x7fe568072310 2024-03-16T18:03:52.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 222943798 0 0) 0x7fe54c003520 con 0x7fe568072310 2024-03-16T18:03:52.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe56815e2c0 msgr2=0x7fe56806d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:52.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe567400640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe56815e2c0 0x7fe56806d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 >> v2:172.21.15.90:3300/0 conn(0x7fe56815dee0 msgr2=0x7fe5680b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe567400640 1 --2- 172.21.15.82:0/51497341 >> v2:172.21.15.90:3300/0 conn(0x7fe56815dee0 0x7fe5680b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.412+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56815ed50 con 0x7fe568072310 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fe560001800 con 0x7fe568072310 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 >> v1:172.21.15.82:6789/0 conn(0x7fe568072310 legacy=0x7fe5680ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe567400640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe56815e2c0 0x7fe54c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe567400640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54c003f40 0x7fe54c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe567400640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe54c004d00 0x7fe54c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56815e790 con 0x7fe54c004d00 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56815ea70 con 0x7fe54c003f40 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c00b980 con 0x7fe56815e2c0 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe56ce00640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe54c004d00 0x7fe54c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe567e00640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe56815e2c0 0x7fe54c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe56ce00640 1 -- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe56815e2c0 msgr2=0x7fe54c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:52.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe55f400640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54c003f40 0x7fe54c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe56ce00640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe56815e2c0 0x7fe54c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe56ce00640 1 -- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54c003f40 msgr2=0x7fe54c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:52.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe56ce00640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54c003f40 0x7fe54c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.413+0000 7fe56ce00640 1 -- 172.21.15.82:0/51497341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c003520 con 0x7fe54c004d00 2024-03-16T18:03:52.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.414+0000 7fe56ce00640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe54c004d00 0x7fe54c005180 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c140 tx=0x7fe54c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ffdc3a51aab674 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:52.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.415+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560005ad0 con 0x7fe54c004d00 2024-03-16T18:03:52.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.415+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe560005c50 con 0x7fe54c004d00 2024-03-16T18:03:52.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.415+0000 7fe567400640 1 -- 172.21.15.82:0/51497341 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56002c7f0 con 0x7fe54c004d00 2024-03-16T18:03:52.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.415+0000 7fe56ee00640 1 -- 172.21.15.82:0/51497341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe54c004d00 msgr2=0x7fe54c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.415+0000 7fe56ee00640 1 --2- 172.21.15.82:0/51497341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe54c004d00 0x7fe54c005180 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe56000c140 tx=0x7fe54c0057f0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.415+0000 7fe56ce00640 1 -- 172.21.15.82:0/51497341 reap_dead start 2024-03-16T18:03:52.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.415+0000 7fe56ee00640 1 -- 172.21.15.82:0/51497341 shutdown_connections 2024-03-16T18:03:52.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.415+0000 7fe56ee00640 1 -- 172.21.15.82:0/51497341 >> 172.21.15.82:0/51497341 conn(0x7fe56806ae70 msgr2=0x7fe56806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:52.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.416+0000 7fe56ee00640 1 -- 172.21.15.82:0/51497341 shutdown_connections 2024-03-16T18:03:52.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.416+0000 7fe56ee00640 1 -- 172.21.15.82:0/51497341 wait complete. 2024-03-16T18:03:52.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.417+0000 7fe56ee00640 1 Processor -- start 2024-03-16T18:03:52.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.417+0000 7fe56ee00640 1 -- start start 2024-03-16T18:03:52.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.418+0000 7fe56ee00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54c004d00 0x7fe5680b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.418+0000 7fe56ee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe568153870 0x7fe568149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.418+0000 7fe56ee00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe5680b1490 0x7fe5680b1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.418+0000 7fe56ee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5681557c0 con 0x7fe568153870 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.418+0000 7fe56ee00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680b1e50 con 0x7fe54c004d00 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.418+0000 7fe56ee00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe568149a90 con 0x7fe5680b1490 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe55f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe568153870 0x7fe568149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe55f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe568153870 0x7fe568149550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49238/0 (socket says 172.21.15.82:49238) 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe55f400640 1 -- 172.21.15.82:0/1655645606 learned_addr learned my addr 172.21.15.82:0/1655645606 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe567e00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54c004d00 0x7fe5680b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe56ce00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe5680b1490 0x7fe5680b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe55f400640 1 -- 172.21.15.82:0/1655645606 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe5680b1490 msgr2=0x7fe5680b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe55f400640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe5680b1490 0x7fe5680b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe55f400640 1 -- 172.21.15.82:0/1655645606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54c004d00 msgr2=0x7fe5680b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe55f400640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54c004d00 0x7fe5680b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe55f400640 1 -- 172.21.15.82:0/1655645606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe568155090 con 0x7fe568153870 2024-03-16T18:03:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.419+0000 7fe55f400640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe568153870 0x7fe568149550 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe550010890 tx=0x7fe550010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.420+0000 7fe564c00640 1 -- 172.21.15.82:0/1655645606 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5500095b0 con 0x7fe568153870 2024-03-16T18:03:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.420+0000 7fe564c00640 1 -- 172.21.15.82:0/1655645606 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55000eb80 con 0x7fe568153870 2024-03-16T18:03:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.420+0000 7fe564c00640 1 -- 172.21.15.82:0/1655645606 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55001c400 con 0x7fe568153870 2024-03-16T18:03:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.420+0000 7fe56ee00640 1 -- 172.21.15.82:0/1655645606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe568153ff0 con 0x7fe568153870 2024-03-16T18:03:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.420+0000 7fe56ee00640 1 -- 172.21.15.82:0/1655645606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5681545b0 con 0x7fe568153870 2024-03-16T18:03:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.421+0000 7fe56ee00640 1 -- 172.21.15.82:0/1655645606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe568067f30 con 0x7fe568153870 2024-03-16T18:03:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.422+0000 7fe564c00640 1 -- 172.21.15.82:0/1655645606 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe5500143f0 con 0x7fe568153870 2024-03-16T18:03:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.423+0000 7fe564c00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe54406b3e0 0x7fe54406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.423+0000 7fe564c00640 1 -- 172.21.15.82:0/1655645606 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe550089ec0 con 0x7fe568153870 2024-03-16T18:03:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.423+0000 7fe567e00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe54406b3e0 0x7fe54406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.424+0000 7fe567e00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe54406b3e0 0x7fe54406d8a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe5580074a0 tx=0x7fe55800b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:52.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.427+0000 7fe564c00640 1 -- 172.21.15.82:0/1655645606 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe550054e00 con 0x7fe568153870 2024-03-16T18:03:52.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.619+0000 7fe56ee00640 1 -- 172.21.15.82:0/1655645606 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe568068860 con 0x7fe54406b3e0 2024-03-16T18:03:52.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.621+0000 7fe564c00640 1 -- 172.21.15.82:0/1655645606 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13496 (secure 0 0 0) 0x7fe568068860 con 0x7fe54406b3e0 2024-03-16T18:03:52.625 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:52.625 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 -- 172.21.15.82:0/1655645606 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe54406b3e0 msgr2=0x7fe54406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe54406b3e0 0x7fe54406d8a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe5580074a0 tx=0x7fe55800b040 comp rx=0 tx=0).stop 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 -- 172.21.15.82:0/1655645606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe568153870 msgr2=0x7fe568149550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe568153870 0x7fe568149550 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe550010890 tx=0x7fe550010d50 comp rx=0 tx=0).stop 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 -- 172.21.15.82:0/1655645606 shutdown_connections 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe54406b3e0 0x7fe54406d8a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe5680b1490 0x7fe5680b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe568153870 0x7fe568149550 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 --2- 172.21.15.82:0/1655645606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe54c004d00 0x7fe5680b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 -- 172.21.15.82:0/1655645606 >> 172.21.15.82:0/1655645606 conn(0x7fe56806ae70 msgr2=0x7fe56815ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:52.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 -- 172.21.15.82:0/1655645606 shutdown_connections 2024-03-16T18:03:52.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.624+0000 7fe56ee00640 1 -- 172.21.15.82:0/1655645606 wait complete. 2024-03-16T18:03:52.648 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-03-16T18:03:52.123515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221976,"kb_used_data":5816,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752744,"statfs":{"total":773094113280,"available":772866809856,"internally_reserved":0,"allocated":5955584,"data_stored":3990528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-780544,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-5,"num_write":-157,"num_write_kb":-819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.024061"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-16T18:03:46.108312+0000","last_change":"2024-03-16T17:57:35.560172+0000","last_active":"2024-03-16T18:03:46.108312+0000","last_peered":"2024-03-16T18:03:46.108312+0000","last_clean":"2024-03-16T18:03:46.108312+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T18:03:46.108312+0000","last_undegraded":"2024-03-16T18:03:46.108312+0000","last_fullsized":"2024-03-16T18:03:46.108312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_clean_scrub_stamp":"2024-03-16T17:57:34.174437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:20:03.039404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542223,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":746546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542223,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":156704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542221,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":161709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542221,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":161141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542222,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":930907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542221,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":156158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":930907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":987136,"data_stored":746456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T18:03:52.648 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-16T18:03:52.648 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T18:03:52.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.856+0000 7f1343200640 1 Processor -- start 2024-03-16T18:03:52.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.856+0000 7f1343200640 1 -- start start 2024-03-16T18:03:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.857+0000 7f1343200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1344143e00 0x7f1344070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.857+0000 7f1343200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f13440cd660 0x7f13440cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.857+0000 7f1343200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13440cf3a0 con 0x7f1344070820 2024-03-16T18:03:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1343200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1344071140 con 0x7f1344143e00 2024-03-16T18:03:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1343200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13440cf610 con 0x7f13440cd660 2024-03-16T18:03:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1341e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1344070820 0x7f134415f080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56040/0 (socket says 172.21.15.82:56040) 2024-03-16T18:03:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1341e00640 1 -- 172.21.15.82:0/1252745041 learned_addr learned my addr 172.21.15.82:0/1252745041 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1339400640 1 --2- 172.21.15.82:0/1252745041 >> v2:172.21.15.82:3300/0 conn(0x7f1344143e00 0x7f1344070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1342800640 1 --2- 172.21.15.82:0/1252745041 >> v2:172.21.15.90:3300/0 conn(0x7f13440cd660 0x7f13440cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1341400640 1 -- 172.21.15.82:0/1252745041 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684854748 0 0) 0x7f13440cf3a0 con 0x7f1344070820 2024-03-16T18:03:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1341400640 1 -- 172.21.15.82:0/1252745041 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1328003520 con 0x7f1344070820 2024-03-16T18:03:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1339400640 1 -- 172.21.15.82:0/1252745041 >> v2:172.21.15.90:3300/0 conn(0x7f13440cd660 msgr2=0x7f13440cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1339400640 1 --2- 172.21.15.82:0/1252745041 >> v2:172.21.15.90:3300/0 conn(0x7f13440cd660 0x7f13440cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1339400640 1 -- 172.21.15.82:0/1252745041 >> v1:172.21.15.82:6789/0 conn(0x7f1344070820 legacy=0x7f134415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.858+0000 7f1339400640 1 -- 172.21.15.82:0/1252745041 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13440cf8f0 con 0x7f1344143e00 2024-03-16T18:03:52.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.860+0000 7f1339400640 1 --2- 172.21.15.82:0/1252745041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344143e00 0x7f1344070c00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f13440cd3b0 tx=0x7f132c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb692d82febfca2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.860+0000 7f1341400640 1 -- 172.21.15.82:0/1252745041 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c02e070 con 0x7f1344143e00 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.860+0000 7f1341400640 1 -- 172.21.15.82:0/1252745041 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c0315a0 con 0x7f1344143e00 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.860+0000 7f1341400640 1 -- 172.21.15.82:0/1252745041 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c031930 con 0x7f1344143e00 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.860+0000 7f1343200640 1 -- 172.21.15.82:0/1252745041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344143e00 msgr2=0x7f1344070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.860+0000 7f1343200640 1 --2- 172.21.15.82:0/1252745041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344143e00 0x7f1344070c00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f13440cd3b0 tx=0x7f132c02b840 comp rx=0 tx=0).stop 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.860+0000 7f1343200640 1 -- 172.21.15.82:0/1252745041 shutdown_connections 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.860+0000 7f1343200640 1 --2- 172.21.15.82:0/1252745041 >> v2:172.21.15.90:3300/0 conn(0x7f13440cd660 0x7f13440cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.860+0000 7f1343200640 1 --2- 172.21.15.82:0/1252745041 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1344143e00 0x7f1344070c00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.860+0000 7f1343200640 1 -- 172.21.15.82:0/1252745041 >> 172.21.15.82:0/1252745041 conn(0x7f134406ae70 msgr2=0x7f134406b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.861+0000 7f1343200640 1 -- 172.21.15.82:0/1252745041 shutdown_connections 2024-03-16T18:03:52.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.861+0000 7f1343200640 1 -- 172.21.15.82:0/1252745041 wait complete. 2024-03-16T18:03:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.862+0000 7f1343200640 1 Processor -- start 2024-03-16T18:03:52.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.862+0000 7f1343200640 1 -- start start 2024-03-16T18:03:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.863+0000 7f1343200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1344070820 0x7f13440b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.863+0000 7f1343200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13440cd660 0x7f13440b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.863+0000 7f1343200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1344143e00 0x7f1344158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.863+0000 7f1343200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134415a270 con 0x7f13440cd660 2024-03-16T18:03:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.863+0000 7f1343200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13441596f0 con 0x7f1344143e00 2024-03-16T18:03:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.863+0000 7f1343200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13441599d0 con 0x7f1344070820 2024-03-16T18:03:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.864+0000 7f1339400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13440cd660 0x7f13440b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.864+0000 7f1339400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13440cd660 0x7f13440b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49258/0 (socket says 172.21.15.82:49258) 2024-03-16T18:03:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.864+0000 7f1339400640 1 -- 172.21.15.82:0/1240021030 learned_addr learned my addr 172.21.15.82:0/1240021030 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:52.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.864+0000 7f1341e00640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1344070820 0x7f13440b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.864+0000 7f1342800640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1344143e00 0x7f1344158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.864+0000 7f1339400640 1 -- 172.21.15.82:0/1240021030 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1344070820 msgr2=0x7f13440b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.864+0000 7f1339400640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1344070820 0x7f13440b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.864+0000 7f1339400640 1 -- 172.21.15.82:0/1240021030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1344143e00 msgr2=0x7f1344158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:52.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.864+0000 7f1339400640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1344143e00 0x7f1344158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:52.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.864+0000 7f1339400640 1 -- 172.21.15.82:0/1240021030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c009400 con 0x7f13440cd660 2024-03-16T18:03:52.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.865+0000 7f1339400640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13440cd660 0x7f13440b3300 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f132c009890 tx=0x7f132c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:52.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.865+0000 7f133aa00640 1 -- 172.21.15.82:0/1240021030 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c032840 con 0x7f13440cd660 2024-03-16T18:03:52.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.865+0000 7f133aa00640 1 -- 172.21.15.82:0/1240021030 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c031f00 con 0x7f13440cd660 2024-03-16T18:03:52.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.865+0000 7f133aa00640 1 -- 172.21.15.82:0/1240021030 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c032be0 con 0x7f13440cd660 2024-03-16T18:03:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.865+0000 7f1343200640 1 -- 172.21.15.82:0/1240021030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1344159cb0 con 0x7f13440cd660 2024-03-16T18:03:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.865+0000 7f1343200640 1 -- 172.21.15.82:0/1240021030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134413f880 con 0x7f13440cd660 2024-03-16T18:03:52.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.866+0000 7f1343200640 1 -- 172.21.15.82:0/1240021030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13440b4d80 con 0x7f13440cd660 2024-03-16T18:03:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.867+0000 7f133aa00640 1 -- 172.21.15.82:0/1240021030 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f132c032d40 con 0x7f13440cd660 2024-03-16T18:03:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.868+0000 7f133aa00640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f132006b3b0 0x7f132006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.868+0000 7f133aa00640 1 -- 172.21.15.82:0/1240021030 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f132c0ab7e0 con 0x7f13440cd660 2024-03-16T18:03:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.868+0000 7f1341e00640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f132006b3b0 0x7f132006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:52.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.869+0000 7f1341e00640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f132006b3b0 0x7f132006d870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1334007fd0 tx=0x7f1334009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:52.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:52.872+0000 7f133aa00640 1 -- 172.21.15.82:0/1240021030 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f132c076740 con 0x7f13440cd660 2024-03-16T18:03:53.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.067+0000 7f1343200640 1 -- 172.21.15.82:0/1240021030 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1344143b50 con 0x7f132006b3b0 2024-03-16T18:03:53.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.068+0000 7f133aa00640 1 -- 172.21.15.82:0/1240021030 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13496 (secure 0 0 0) 0x7f13340013f0 con 0x7f132006b3b0 2024-03-16T18:03:53.073 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:53.073 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T18:03:53.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.071+0000 7f1343200640 1 -- 172.21.15.82:0/1240021030 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f132006b3b0 msgr2=0x7f132006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:53.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.071+0000 7f1343200640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f132006b3b0 0x7f132006d870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1334007fd0 tx=0x7f1334009040 comp rx=0 tx=0).stop 2024-03-16T18:03:53.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.071+0000 7f1343200640 1 -- 172.21.15.82:0/1240021030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13440cd660 msgr2=0x7f13440b3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.071+0000 7f1343200640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13440cd660 0x7f13440b3300 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f132c009890 tx=0x7f132c0046c0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.071+0000 7f1343200640 1 -- 172.21.15.82:0/1240021030 shutdown_connections 2024-03-16T18:03:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.071+0000 7f1343200640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f132006b3b0 0x7f132006d870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.071+0000 7f1343200640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1344143e00 0x7f1344158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.071+0000 7f1343200640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13440cd660 0x7f13440b3300 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.071+0000 7f1343200640 1 --2- 172.21.15.82:0/1240021030 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f1344070820 0x7f13440b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.071+0000 7f1343200640 1 -- 172.21.15.82:0/1240021030 >> 172.21.15.82:0/1240021030 conn(0x7f134406ae70 msgr2=0x7f13440b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.072+0000 7f1343200640 1 -- 172.21.15.82:0/1240021030 shutdown_connections 2024-03-16T18:03:53.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.072+0000 7f1343200640 1 -- 172.21.15.82:0/1240021030 wait complete. 2024-03-16T18:03:53.092 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-03-16T18:03:52.123515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221976,"kb_used_data":5816,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754752744,"statfs":{"total":773094113280,"available":772866809856,"internally_reserved":0,"allocated":5955584,"data_stored":3990528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-780544,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-5,"num_write":-157,"num_write_kb":-819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.024061"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-16T18:03:46.108312+0000","last_change":"2024-03-16T17:57:35.560172+0000","last_active":"2024-03-16T18:03:46.108312+0000","last_peered":"2024-03-16T18:03:46.108312+0000","last_clean":"2024-03-16T18:03:46.108312+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T18:03:46.108312+0000","last_undegraded":"2024-03-16T18:03:46.108312+0000","last_fullsized":"2024-03-16T18:03:46.108312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_clean_scrub_stamp":"2024-03-16T17:57:34.174437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:20:03.039404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542223,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":991232,"data_stored":746546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542223,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":156704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542221,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":161709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542221,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":161141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542222,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":930907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542221,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":156158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542222,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28172,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343668,"statfs":{"total":96636764160,"available":96607916032,"internally_reserved":0,"allocated":1179648,"data_stored":930907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":987136,"data_stored":746456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T18:03:53.093 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T18:03:53.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.301+0000 7ffa6ee00640 1 Processor -- start 2024-03-16T18:03:53.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.301+0000 7ffa6ee00640 1 -- start start 2024-03-16T18:03:53.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.303+0000 7ffa6ee00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa68144480 0x7ffa6815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:53.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.303+0000 7ffa6ee00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa680ce670 con 0x7ffa681440a0 2024-03-16T18:03:53.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.303+0000 7ffa6ee00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa680ccff0 con 0x7ffa6806e250 2024-03-16T18:03:53.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.303+0000 7ffa6ee00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6815f1a0 con 0x7ffa68144480 2024-03-16T18:03:53.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.303+0000 7ffa6da00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffa681440a0 0x7ffa6806db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56054/0 (socket says 172.21.15.82:56054) 2024-03-16T18:03:53.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.303+0000 7ffa6da00640 1 -- 172.21.15.82:0/2267873482 learned_addr learned my addr 172.21.15.82:0/2267873482 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:53.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.303+0000 7ffa6d000640 1 --2- 172.21.15.82:0/2267873482 >> v2:172.21.15.82:3300/0 conn(0x7ffa68144480 0x7ffa6815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:53.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.304+0000 7ffa67e00640 1 -- 172.21.15.82:0/2267873482 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626917827 0 0) 0x7ffa680ce670 con 0x7ffa681440a0 2024-03-16T18:03:53.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.304+0000 7ffa6d000640 1 -- 172.21.15.82:0/2267873482 >> v1:172.21.15.90:6789/0 conn(0x7ffa6806e250 legacy=0x7ffa6815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:53.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.304+0000 7ffa6d000640 1 -- 172.21.15.82:0/2267873482 >> v1:172.21.15.82:6789/0 conn(0x7ffa681440a0 legacy=0x7ffa6806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:53.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.304+0000 7ffa6d000640 1 -- 172.21.15.82:0/2267873482 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa680cc230 con 0x7ffa68144480 2024-03-16T18:03:53.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.305+0000 7ffa6d000640 1 --2- 172.21.15.82:0/2267873482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa68144480 0x7ffa6815e430 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffa540097a0 tx=0x7ffa5402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d390b8823648e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:53.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.305+0000 7ffa67e00640 1 -- 172.21.15.82:0/2267873482 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54031830 con 0x7ffa68144480 2024-03-16T18:03:53.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.305+0000 7ffa67e00640 1 -- 172.21.15.82:0/2267873482 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa54031990 con 0x7ffa68144480 2024-03-16T18:03:53.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.305+0000 7ffa67e00640 1 -- 172.21.15.82:0/2267873482 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa54031d20 con 0x7ffa68144480 2024-03-16T18:03:53.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.306+0000 7ffa6ee00640 1 -- 172.21.15.82:0/2267873482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa68144480 msgr2=0x7ffa6815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:53.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.306+0000 7ffa6ee00640 1 --2- 172.21.15.82:0/2267873482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa68144480 0x7ffa6815e430 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffa540097a0 tx=0x7ffa5402b8d0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.306+0000 7ffa6ee00640 1 -- 172.21.15.82:0/2267873482 shutdown_connections 2024-03-16T18:03:53.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.306+0000 7ffa6ee00640 1 --2- 172.21.15.82:0/2267873482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa68144480 0x7ffa6815e430 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.306+0000 7ffa6ee00640 1 -- 172.21.15.82:0/2267873482 >> 172.21.15.82:0/2267873482 conn(0x7ffa6806ae70 msgr2=0x7ffa6806b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:53.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.306+0000 7ffa6ee00640 1 -- 172.21.15.82:0/2267873482 shutdown_connections 2024-03-16T18:03:53.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.306+0000 7ffa6ee00640 1 -- 172.21.15.82:0/2267873482 wait complete. 2024-03-16T18:03:53.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.308+0000 7ffa6ee00640 1 Processor -- start 2024-03-16T18:03:53.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.308+0000 7ffa6ee00640 1 -- start start 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6ee00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa6806e250 0x7ffa6815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6ee00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffa681440a0 0x7ffa6813e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6ee00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa68144480 0x7ffa68141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6ee00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa680b2e20 con 0x7ffa6806e250 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6ee00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6813e8f0 con 0x7ffa68144480 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6ee00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa68142460 con 0x7ffa681440a0 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa6806e250 0x7ffa6815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6da00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa6806e250 0x7ffa6815a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49270/0 (socket says 172.21.15.82:49270) 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6da00640 1 -- 172.21.15.82:0/3859813939 learned_addr learned my addr 172.21.15.82:0/3859813939 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6e400640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa68144480 0x7ffa68141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6d000640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffa681440a0 0x7ffa6813e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:53.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6da00640 1 -- 172.21.15.82:0/3859813939 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffa681440a0 msgr2=0x7ffa6813e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:53.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6da00640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffa681440a0 0x7ffa6813e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6da00640 1 -- 172.21.15.82:0/3859813939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa68144480 msgr2=0x7ffa68141f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:53.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6da00640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa68144480 0x7ffa68141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.309+0000 7ffa6da00640 1 -- 172.21.15.82:0/3859813939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa54009400 con 0x7ffa6806e250 2024-03-16T18:03:53.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.310+0000 7ffa6da00640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa6806e250 0x7ffa6815a3d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00c3f0 tx=0x7ffa5c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:53.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.310+0000 7ffa65600640 1 -- 172.21.15.82:0/3859813939 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0074e0 con 0x7ffa6806e250 2024-03-16T18:03:53.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.310+0000 7ffa65600640 1 -- 172.21.15.82:0/3859813939 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c007640 con 0x7ffa6806e250 2024-03-16T18:03:53.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.310+0000 7ffa6ee00640 1 -- 172.21.15.82:0/3859813939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa680724e0 con 0x7ffa6806e250 2024-03-16T18:03:53.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.310+0000 7ffa65600640 1 -- 172.21.15.82:0/3859813939 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c004a00 con 0x7ffa6806e250 2024-03-16T18:03:53.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.310+0000 7ffa6ee00640 1 -- 172.21.15.82:0/3859813939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa680b27d0 con 0x7ffa6806e250 2024-03-16T18:03:53.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.311+0000 7ffa6ee00640 1 -- 172.21.15.82:0/3859813939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6804f250 con 0x7ffa6806e250 2024-03-16T18:03:53.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.312+0000 7ffa65600640 1 -- 172.21.15.82:0/3859813939 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7ffa5c004b60 con 0x7ffa6806e250 2024-03-16T18:03:53.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.313+0000 7ffa65600640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ffa4806b3e0 0x7ffa4806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:53.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.313+0000 7ffa65600640 1 -- 172.21.15.82:0/3859813939 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffa5c0897a0 con 0x7ffa6806e250 2024-03-16T18:03:53.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.313+0000 7ffa6d000640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ffa4806b3e0 0x7ffa4806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:53.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.314+0000 7ffa6d000640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ffa4806b3e0 0x7ffa4806d8a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffa54036000 tx=0x7ffa54008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:53.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.317+0000 7ffa65600640 1 -- 172.21.15.82:0/3859813939 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffa5c054660 con 0x7ffa6806e250 2024-03-16T18:03:53.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.514+0000 7ffa6ee00640 1 -- 172.21.15.82:0/3859813939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa680adea0 con 0x7ffa6806e250 2024-03-16T18:03:53.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.515+0000 7ffa65600640 1 -- 172.21.15.82:0/3859813939 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+12984 (secure 0 0 0) 0x7ffa5c057d20 con 0x7ffa6806e250 2024-03-16T18:03:53.519 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:03:53.519 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":37,"fsid":"08082c2e-06a4-4a16-ab2e-9e9e244a6661","created":"2024-03-16T17:57:21.121981+0000","modified":"2024-03-16T18:03:49.289334+0000","last_up_change":"2024-03-16T17:57:33.170235+0000","last_in_change":"2024-03-16T17:57:24.823529+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T17:57:34.080848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3e9a1199-4782-4263-98ef-1885f932bc89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6801","nonce":2810023632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6803","nonce":2810023632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6807","nonce":2810023632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":2810023632},{"type":"v1","addr":"172.21.15.82:6805","nonce":2810023632}]},"public_addr":"172.21.15.82:6801/2810023632","cluster_addr":"172.21.15.82:6803/2810023632","heartbeat_back_addr":"172.21.15.82:6807/2810023632","heartbeat_front_addr":"172.21.15.82:6805/2810023632","state":["exists","up"]},{"osd":1,"uuid":"57f8eb0b-64d2-4879-96ce-c786ff39aaf0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6809","nonce":1061186346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6811","nonce":1061186346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6815","nonce":1061186346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":1061186346},{"type":"v1","addr":"172.21.15.82:6813","nonce":1061186346}]},"public_addr":"172.21.15.82:6809/1061186346","cluster_addr":"172.21.15.82:6811/1061186346","heartbeat_back_addr":"172.21.15.82:6815/1061186346","heartbeat_front_addr":"172.21.15.82:6813/1061186346","state":["exists","up"]},{"osd":2,"uuid":"93d45d8a-6281-41da-a4dd-bcbb25bb24a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6825","nonce":1740405378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6827","nonce":1740405378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6831","nonce":1740405378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":1740405378},{"type":"v1","addr":"172.21.15.82:6829","nonce":1740405378}]},"public_addr":"172.21.15.82:6825/1740405378","cluster_addr":"172.21.15.82:6827/1740405378","heartbeat_back_addr":"172.21.15.82:6831/1740405378","heartbeat_front_addr":"172.21.15.82:6829/1740405378","state":["exists","up"]},{"osd":3,"uuid":"61bf727a-c14b-4cfd-97e0-d4156d1cd362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6817","nonce":4087215098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6819","nonce":4087215098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6823","nonce":4087215098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":4087215098},{"type":"v1","addr":"172.21.15.82:6821","nonce":4087215098}]},"public_addr":"172.21.15.82:6817/4087215098","cluster_addr":"172.21.15.82:6819/4087215098","heartbeat_back_addr":"172.21.15.82:6823/4087215098","heartbeat_front_addr":"172.21.15.82:6821/4087215098","state":["exists","up"]},{"osd":4,"uuid":"fcb62abd-4b1f-4f86-a989-88111ec0e595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6801","nonce":3078143209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6803","nonce":3078143209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6807","nonce":3078143209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":3078143209},{"type":"v1","addr":"172.21.15.90:6805","nonce":3078143209}]},"public_addr":"172.21.15.90:6801/3078143209","cluster_addr":"172.21.15.90:6803/3078143209","heartbeat_back_addr":"172.21.15.90:6807/3078143209","heartbeat_front_addr":"172.21.15.90:6805/3078143209","state":["exists","up"]},{"osd":5,"uuid":"e38bdb1d-ad07-4368-84b8-a6e6afa0ae93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6809","nonce":2407665501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6811","nonce":2407665501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6815","nonce":2407665501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2407665501},{"type":"v1","addr":"172.21.15.90:6813","nonce":2407665501}]},"public_addr":"172.21.15.90:6809/2407665501","cluster_addr":"172.21.15.90:6811/2407665501","heartbeat_back_addr":"172.21.15.90:6815/2407665501","heartbeat_front_addr":"172.21.15.90:6813/2407665501","state":["exists","up"]},{"osd":6,"uuid":"e08850eb-1b52-4cb7-bbcb-265031c59075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6818","nonce":1201371759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6822","nonce":1201371759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6830","nonce":1201371759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1201371759},{"type":"v1","addr":"172.21.15.90:6826","nonce":1201371759}]},"public_addr":"172.21.15.90:6818/1201371759","cluster_addr":"172.21.15.90:6822/1201371759","heartbeat_back_addr":"172.21.15.90:6830/1201371759","heartbeat_front_addr":"172.21.15.90:6826/1201371759","state":["exists","up"]},{"osd":7,"uuid":"c948154a-8b9b-4a9a-967a-615651402176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6817","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6819","nonce":3392471023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6821","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6823","nonce":3392471023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6829","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6831","nonce":3392471023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6825","nonce":3392471023},{"type":"v1","addr":"172.21.15.90:6827","nonce":3392471023}]},"public_addr":"172.21.15.90:6819/3392471023","cluster_addr":"172.21.15.90:6823/3392471023","heartbeat_back_addr":"172.21.15.90:6831/3392471023","heartbeat_front_addr":"172.21.15.90:6827/3392471023","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.107998+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.073752+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.196356+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.134997+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.362302+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.098514+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:30.109447+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T17:57:32.278208+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6835/2421794786":"2024-03-17T18:03:46.094677+0000","172.21.15.90:6833/2421794786":"2024-03-17T18:03:46.094677+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 -- 172.21.15.82:0/3859813939 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ffa4806b3e0 msgr2=0x7ffa4806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ffa4806b3e0 0x7ffa4806d8a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffa54036000 tx=0x7ffa54008030 comp rx=0 tx=0).stop 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 -- 172.21.15.82:0/3859813939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa6806e250 msgr2=0x7ffa6815a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa6806e250 0x7ffa6815a3d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00c3f0 tx=0x7ffa5c00c8b0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 -- 172.21.15.82:0/3859813939 shutdown_connections 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ffa4806b3e0 0x7ffa4806d8a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa68144480 0x7ffa68141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ffa681440a0 0x7ffa6813e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 --2- 172.21.15.82:0/3859813939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa6806e250 0x7ffa6815a3d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 -- 172.21.15.82:0/3859813939 >> 172.21.15.82:0/3859813939 conn(0x7ffa6806ae70 msgr2=0x7ffa680b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 -- 172.21.15.82:0/3859813939 shutdown_connections 2024-03-16T18:03:53.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:53.518+0000 7ffa6ee00640 1 -- 172.21.15.82:0/3859813939 wait complete. 2024-03-16T18:03:54.541 INFO:tasks.ceph:Scrubbing osd.0 2024-03-16T18:03:54.542 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-16T18:03:54.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.763+0000 7fe10b600640 1 Processor -- start 2024-03-16T18:03:54.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.763+0000 7fe10b600640 1 -- start start 2024-03-16T18:03:54.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.765+0000 7fe10b600640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe1040ccf30 0x7fe10415a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:54.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.765+0000 7fe10b600640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fe1040b4270 0x7fe1040b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:54.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.765+0000 7fe10b600640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe104071f40 con 0x7fe1040b2a40 2024-03-16T18:03:54.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.765+0000 7fe10b600640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1040ccb50 con 0x7fe1040ccf30 2024-03-16T18:03:54.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.765+0000 7fe10b600640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe104071330 con 0x7fe1040b4270 2024-03-16T18:03:54.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.765+0000 7fe108c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe1040ccf30 0x7fe10415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:54.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.765+0000 7fe108c00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe1040ccf30 0x7fe10415a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49280/0 (socket says 172.21.15.82:49280) 2024-03-16T18:03:54.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.765+0000 7fe108c00640 1 -- 172.21.15.82:0/778558181 learned_addr learned my addr 172.21.15.82:0/778558181 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:54.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.766+0000 7fe109600640 1 --2- 172.21.15.82:0/778558181 >> v2:172.21.15.90:3300/0 conn(0x7fe1040b4270 0x7fe1040b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:54.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.766+0000 7fe108c00640 1 -- 172.21.15.82:0/778558181 >> v2:172.21.15.90:3300/0 conn(0x7fe1040b4270 msgr2=0x7fe1040b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:54.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.766+0000 7fe108c00640 1 --2- 172.21.15.82:0/778558181 >> v2:172.21.15.90:3300/0 conn(0x7fe1040b4270 0x7fe1040b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:54.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.766+0000 7fe108c00640 1 -- 172.21.15.82:0/778558181 >> v1:172.21.15.90:6789/0 conn(0x7fe1040b2a40 legacy=0x7fe1040cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:54.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.766+0000 7fe108c00640 1 -- 172.21.15.82:0/778558181 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe104071610 con 0x7fe1040ccf30 2024-03-16T18:03:54.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.767+0000 7fe108c00640 1 --2- 172.21.15.82:0/778558181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1040ccf30 0x7fe10415a410 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe1040732e0 tx=0x7fe0f402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98239816b1202340 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:54.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.767+0000 7fe103400640 1 -- 172.21.15.82:0/778558181 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f40317e0 con 0x7fe1040ccf30 2024-03-16T18:03:54.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.767+0000 7fe103400640 1 -- 172.21.15.82:0/778558181 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f4031940 con 0x7fe1040ccf30 2024-03-16T18:03:54.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.767+0000 7fe103400640 1 -- 172.21.15.82:0/778558181 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4031cd0 con 0x7fe1040ccf30 2024-03-16T18:03:54.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.768+0000 7fe10b600640 1 -- 172.21.15.82:0/778558181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1040ccf30 msgr2=0x7fe10415a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:54.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.768+0000 7fe10b600640 1 --2- 172.21.15.82:0/778558181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1040ccf30 0x7fe10415a410 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe1040732e0 tx=0x7fe0f402b880 comp rx=0 tx=0).stop 2024-03-16T18:03:54.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.768+0000 7fe10b600640 1 -- 172.21.15.82:0/778558181 shutdown_connections 2024-03-16T18:03:54.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.768+0000 7fe10b600640 1 --2- 172.21.15.82:0/778558181 >> v2:172.21.15.90:3300/0 conn(0x7fe1040b4270 0x7fe1040b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:54.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.768+0000 7fe10b600640 1 --2- 172.21.15.82:0/778558181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1040ccf30 0x7fe10415a410 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:54.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.768+0000 7fe10b600640 1 -- 172.21.15.82:0/778558181 >> 172.21.15.82:0/778558181 conn(0x7fe1040b1740 msgr2=0x7fe1040b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:54.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.768+0000 7fe10b600640 1 -- 172.21.15.82:0/778558181 shutdown_connections 2024-03-16T18:03:54.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.768+0000 7fe10b600640 1 -- 172.21.15.82:0/778558181 wait complete. 2024-03-16T18:03:54.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.770+0000 7fe10b600640 1 Processor -- start 2024-03-16T18:03:54.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.770+0000 7fe10b600640 1 -- start start 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.771+0000 7fe10b600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1040b2a40 0x7fe10406d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.771+0000 7fe10b600640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1040b4270 0x7fe10413a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.771+0000 7fe10b600640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1040ccf30 0x7fe10414e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.771+0000 7fe10b600640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10406d760 con 0x7fe1040b2a40 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.771+0000 7fe10b600640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10415b400 con 0x7fe1040ccf30 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.771+0000 7fe10b600640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10415a7f0 con 0x7fe1040b4270 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.771+0000 7fe108c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1040b2a40 0x7fe10406d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.771+0000 7fe108c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1040b2a40 0x7fe10406d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49296/0 (socket says 172.21.15.82:49296) 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.771+0000 7fe108c00640 1 -- 172.21.15.82:0/2856238153 learned_addr learned my addr 172.21.15.82:0/2856238153 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe108c00640 1 -- 172.21.15.82:0/2856238153 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1040b4270 msgr2=0x7fe10413a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe108c00640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe1040b4270 0x7fe10413a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe108c00640 1 -- 172.21.15.82:0/2856238153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1040ccf30 msgr2=0x7fe10414e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe109600640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1040ccf30 0x7fe10414e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe108c00640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1040ccf30 0x7fe10414e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe108c00640 1 -- 172.21.15.82:0/2856238153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f40094d0 con 0x7fe1040b2a40 2024-03-16T18:03:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe108c00640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1040b2a40 0x7fe10406d220 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe0f402b8c0 tx=0x7fe0f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe100c00640 1 -- 172.21.15.82:0/2856238153 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4032880 con 0x7fe1040b2a40 2024-03-16T18:03:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe100c00640 1 -- 172.21.15.82:0/2856238153 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f40329e0 con 0x7fe1040b2a40 2024-03-16T18:03:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe100c00640 1 -- 172.21.15.82:0/2856238153 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4032d90 con 0x7fe1040b2a40 2024-03-16T18:03:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10415aad0 con 0x7fe1040b2a40 2024-03-16T18:03:54.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.772+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10415b090 con 0x7fe1040b2a40 2024-03-16T18:03:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.773+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1040b4c40 con 0x7fe1040b2a40 2024-03-16T18:03:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.774+0000 7fe100c00640 1 -- 172.21.15.82:0/2856238153 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe0f404e450 con 0x7fe1040b2a40 2024-03-16T18:03:54.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.775+0000 7fe100c00640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b400 0x7fe0e006d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:54.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.775+0000 7fe100c00640 1 -- 172.21.15.82:0/2856238153 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe0f40ab9a0 con 0x7fe1040b2a40 2024-03-16T18:03:54.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.775+0000 7fe103e00640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b400 0x7fe0e006d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:54.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.775+0000 7fe100c00640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] conn(0x7fe0e00712c0 0x7fe0e00736e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:54.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.775+0000 7fe100c00640 1 -- 172.21.15.82:0/2856238153 --> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe0e0073d90 con 0x7fe0e00712c0 2024-03-16T18:03:54.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.775+0000 7fe100c00640 1 -- 172.21.15.82:0/2856238153 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0f40b02c0 con 0x7fe1040b2a40 2024-03-16T18:03:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.776+0000 7fe109600640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] conn(0x7fe0e00712c0 0x7fe0e00736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.776+0000 7fe103e00640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b400 0x7fe0e006d8c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec006fd0 tx=0x7fe0ec006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.776+0000 7fe109600640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] conn(0x7fe0e00712c0 0x7fe0e00736e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:54.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.777+0000 7fe100c00640 1 -- 172.21.15.82:0/2856238153 <== osd.0 v2:172.21.15.82:6800/2810023632 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe0e0073d90 con 0x7fe0e00712c0 2024-03-16T18:03:54.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.792+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 --> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe10414a6c0 con 0x7fe0e00712c0 2024-03-16T18:03:54.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.792+0000 7fe100c00640 1 -- 172.21.15.82:0/2856238153 <== osd.0 v2:172.21.15.82:6800/2810023632 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe10414a6c0 con 0x7fe0e00712c0 2024-03-16T18:03:54.796 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T18:03:54.796 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T18:03:54.796 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] conn(0x7fe0e00712c0 msgr2=0x7fe0e00736e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe10b600640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:6800/2810023632,v1:172.21.15.82:6801/2810023632] conn(0x7fe0e00712c0 0x7fe0e00736e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b400 msgr2=0x7fe0e006d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe10b600640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe0e006b400 0x7fe0e006d8c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec006fd0 tx=0x7fe0ec006bc0 comp rx=0 tx=0).stop 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1040b2a40 msgr2=0x7fe10406d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe10b600640 1 --2- 172.21.15.82:0/2856238153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1040b2a40 0x7fe10406d220 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe0f402b8c0 tx=0x7fe0f40046c0 comp rx=0 tx=0).stop 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe109600640 1 -- 172.21.15.82:0/2856238153 reap_dead start 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 shutdown_connections 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 >> 172.21.15.82:0/2856238153 conn(0x7fe1040b1740 msgr2=0x7fe10414f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 shutdown_connections 2024-03-16T18:03:54.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:54.793+0000 7fe10b600640 1 -- 172.21.15.82:0/2856238153 wait complete. 2024-03-16T18:03:54.814 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-16T18:03:55.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.024+0000 7f767f200640 1 Processor -- start 2024-03-16T18:03:55.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.024+0000 7f767f200640 1 -- start start 2024-03-16T18:03:55.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.025+0000 7f767f200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f76780aec80 0x7f76780ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.026+0000 7f767f200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f76781450e0 0x7f767815f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.026+0000 7f767f200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767806ded0 con 0x7f76780ce850 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.026+0000 7f767f200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76780af060 con 0x7f76780aec80 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.026+0000 7f767f200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7678072c10 con 0x7f76781450e0 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.026+0000 7f7677e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f76780aec80 0x7f76780ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.026+0000 7f7677e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f76780aec80 0x7f76780ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60106/0 (socket says 172.21.15.82:60106) 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.026+0000 7f7677e00640 1 -- 172.21.15.82:0/4243056928 learned_addr learned my addr 172.21.15.82:0/4243056928 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.026+0000 7f767d200640 1 --2- 172.21.15.82:0/4243056928 >> v2:172.21.15.90:3300/0 conn(0x7f76781450e0 0x7f767815f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.027+0000 7f7677e00640 1 -- 172.21.15.82:0/4243056928 >> v2:172.21.15.90:3300/0 conn(0x7f76781450e0 msgr2=0x7f767815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.027+0000 7f7677e00640 1 --2- 172.21.15.82:0/4243056928 >> v2:172.21.15.90:3300/0 conn(0x7f76781450e0 0x7f767815f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.027+0000 7f7677e00640 1 -- 172.21.15.82:0/4243056928 >> v1:172.21.15.90:6789/0 conn(0x7f76780ce850 legacy=0x7f7678070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.027+0000 7f7677e00640 1 -- 172.21.15.82:0/4243056928 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7678160730 con 0x7f76780aec80 2024-03-16T18:03:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.028+0000 7f7677e00640 1 --2- 172.21.15.82:0/4243056928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76780aec80 0x7f76780ce310 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f76780708d0 tx=0x7f766802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83045807984f9a9a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.029+0000 7f7677400640 1 -- 172.21.15.82:0/4243056928 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766802ca80 con 0x7f76780aec80 2024-03-16T18:03:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.029+0000 7f7677400640 1 -- 172.21.15.82:0/4243056928 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766802cbe0 con 0x7f76780aec80 2024-03-16T18:03:55.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.029+0000 7f7677400640 1 -- 172.21.15.82:0/4243056928 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76680343f0 con 0x7f76780aec80 2024-03-16T18:03:55.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.029+0000 7f767f200640 1 -- 172.21.15.82:0/4243056928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76780aec80 msgr2=0x7f76780ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.029+0000 7f767f200640 1 --2- 172.21.15.82:0/4243056928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76780aec80 0x7f76780ce310 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f76780708d0 tx=0x7f766802bee0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.030+0000 7f767f200640 1 -- 172.21.15.82:0/4243056928 shutdown_connections 2024-03-16T18:03:55.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.030+0000 7f767f200640 1 --2- 172.21.15.82:0/4243056928 >> v2:172.21.15.90:3300/0 conn(0x7f76781450e0 0x7f767815f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.030+0000 7f767f200640 1 --2- 172.21.15.82:0/4243056928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76780aec80 0x7f76780ce310 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.030+0000 7f767f200640 1 -- 172.21.15.82:0/4243056928 >> 172.21.15.82:0/4243056928 conn(0x7f76780b17b0 msgr2=0x7f76780b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:55.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.030+0000 7f767f200640 1 -- 172.21.15.82:0/4243056928 shutdown_connections 2024-03-16T18:03:55.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.030+0000 7f767f200640 1 -- 172.21.15.82:0/4243056928 wait complete. 2024-03-16T18:03:55.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.031+0000 7f767f200640 1 Processor -- start 2024-03-16T18:03:55.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.032+0000 7f767f200640 1 -- start start 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.032+0000 7f767f200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76780aec80 0x7f767815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.032+0000 7f767f200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76781450e0 0x7f76781e5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.032+0000 7f767f200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76781e5ba0 0x7f767815e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.032+0000 7f767f200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76781e60b0 con 0x7f76781450e0 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.032+0000 7f767f200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76781e6390 con 0x7f76780aec80 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.032+0000 7f767f200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76781e6670 con 0x7f76781e5ba0 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.033+0000 7f766f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76781450e0 0x7f76781e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.033+0000 7f766f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76781450e0 0x7f76781e5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60112/0 (socket says 172.21.15.82:60112) 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.033+0000 7f766f400640 1 -- 172.21.15.82:0/1802228556 learned_addr learned my addr 172.21.15.82:0/1802228556 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.033+0000 7f7677e00640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76780aec80 0x7f767815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.033+0000 7f767d200640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76781e5ba0 0x7f767815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.033+0000 7f766f400640 1 -- 172.21.15.82:0/1802228556 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76781e5ba0 msgr2=0x7f767815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.033+0000 7f766f400640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76781e5ba0 0x7f767815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.033+0000 7f766f400640 1 -- 172.21.15.82:0/1802228556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76780aec80 msgr2=0x7f767815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.033+0000 7f766f400640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76780aec80 0x7f767815bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.033+0000 7f766f400640 1 -- 172.21.15.82:0/1802228556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7668009b30 con 0x7f76781450e0 2024-03-16T18:03:55.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.034+0000 7f766f400640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76781450e0 0x7f76781e5660 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f766000c3c0 tx=0x7f766000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:55.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.034+0000 7f7674c00640 1 -- 172.21.15.82:0/1802228556 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76600052d0 con 0x7f76781450e0 2024-03-16T18:03:55.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.034+0000 7f7674c00640 1 -- 172.21.15.82:0/1802228556 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7660005450 con 0x7f76781450e0 2024-03-16T18:03:55.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.034+0000 7f7674c00640 1 -- 172.21.15.82:0/1802228556 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7660007260 con 0x7f76781450e0 2024-03-16T18:03:55.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.034+0000 7f767f200640 1 -- 172.21.15.82:0/1802228556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76781e7d50 con 0x7f76781450e0 2024-03-16T18:03:55.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.034+0000 7f767f200640 1 -- 172.21.15.82:0/1802228556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76781e74e0 con 0x7f76781450e0 2024-03-16T18:03:55.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.035+0000 7f767f200640 1 -- 172.21.15.82:0/1802228556 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76781e7670 con 0x7f76781450e0 2024-03-16T18:03:55.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.036+0000 7f7674c00640 1 -- 172.21.15.82:0/1802228556 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f76600073c0 con 0x7f76781450e0 2024-03-16T18:03:55.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.037+0000 7f7674c00640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f765406b3b0 0x7f765406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.038+0000 7f7674c00640 1 -- 172.21.15.82:0/1802228556 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7660088b10 con 0x7f76781450e0 2024-03-16T18:03:55.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.038+0000 7f7677e00640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f765406b3b0 0x7f765406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.038+0000 7f7677e00640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f765406b3b0 0x7f765406d870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7668035000 tx=0x7f7668030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:55.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.042+0000 7f7674c00640 1 -- 172.21.15.82:0/1802228556 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7660053a70 con 0x7f76781450e0 2024-03-16T18:03:55.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.240+0000 7f767f200640 1 -- 172.21.15.82:0/1802228556 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f7678072620 con 0x7f765406b3b0 2024-03-16T18:03:55.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.241+0000 7f7674c00640 1 -- 172.21.15.82:0/1802228556 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7668044090 con 0x7f765406b3b0 2024-03-16T18:03:55.244 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 0 to deep-scrub 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 -- 172.21.15.82:0/1802228556 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f765406b3b0 msgr2=0x7f765406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f765406b3b0 0x7f765406d870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7668035000 tx=0x7f7668030000 comp rx=0 tx=0).stop 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 -- 172.21.15.82:0/1802228556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76781450e0 msgr2=0x7f76781e5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76781450e0 0x7f76781e5660 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f766000c3c0 tx=0x7f766000c880 comp rx=0 tx=0).stop 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 -- 172.21.15.82:0/1802228556 shutdown_connections 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f765406b3b0 0x7f765406d870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f76781e5ba0 0x7f767815e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76781450e0 0x7f76781e5660 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 --2- 172.21.15.82:0/1802228556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76780aec80 0x7f767815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 -- 172.21.15.82:0/1802228556 >> 172.21.15.82:0/1802228556 conn(0x7f76780b17b0 msgr2=0x7f76780cf120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:55.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 -- 172.21.15.82:0/1802228556 shutdown_connections 2024-03-16T18:03:55.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.244+0000 7f767f200640 1 -- 172.21.15.82:0/1802228556 wait complete. 2024-03-16T18:03:55.268 INFO:tasks.ceph:Scrubbing osd.1 2024-03-16T18:03:55.268 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-16T18:03:55.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.484+0000 7ff3bb800640 1 Processor -- start 2024-03-16T18:03:55.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.484+0000 7ff3bb800640 1 -- start start 2024-03-16T18:03:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3bb800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3b4144ae0 0x7ff3b406d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3bb800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff3b406d9d0 0x7ff3b4071550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3bb800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7ff3b4071a90 0x7ff3b415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3bb800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b406de30 con 0x7ff3b406d9d0 2024-03-16T18:03:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3bb800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b40cdca0 con 0x7ff3b4071a90 2024-03-16T18:03:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3bb800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b4144ec0 con 0x7ff3b4144ae0 2024-03-16T18:03:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3b3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff3b406d9d0 0x7ff3b4071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3b3e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff3b406d9d0 0x7ff3b4071550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60114/0 (socket says 172.21.15.82:60114) 2024-03-16T18:03:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3b3e00640 1 -- 172.21.15.82:0/2843319256 learned_addr learned my addr 172.21.15.82:0/2843319256 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3b9800640 1 --2- 172.21.15.82:0/2843319256 >> v2:172.21.15.90:3300/0 conn(0x7ff3b4071a90 0x7ff3b415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.486+0000 7ff3b8e00640 1 --2- 172.21.15.82:0/2843319256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3b4144ae0 0x7ff3b406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.487+0000 7ff3b3e00640 1 -- 172.21.15.82:0/2843319256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3b4144ae0 msgr2=0x7ff3b406d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.487+0000 7ff3b3e00640 1 --2- 172.21.15.82:0/2843319256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3b4144ae0 0x7ff3b406d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.487+0000 7ff3b3e00640 1 -- 172.21.15.82:0/2843319256 >> v2:172.21.15.90:3300/0 conn(0x7ff3b4071a90 msgr2=0x7ff3b415f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.487+0000 7ff3b3e00640 1 --2- 172.21.15.82:0/2843319256 >> v2:172.21.15.90:3300/0 conn(0x7ff3b4071a90 0x7ff3b415f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.487+0000 7ff3b3e00640 1 -- 172.21.15.82:0/2843319256 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b40722d0 con 0x7ff3b406d9d0 2024-03-16T18:03:55.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.487+0000 7ff3b8e00640 1 --2- 172.21.15.82:0/2843319256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3b4144ae0 0x7ff3b406d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:55.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.488+0000 7ff3b3e00640 1 --2- 172.21.15.82:0/2843319256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3b406d9d0 0x7ff3b4071550 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40cd640 tx=0x7ff3a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520f456b08f12cb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:55.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.488+0000 7ff3b3400640 1 -- 172.21.15.82:0/2843319256 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0031870 con 0x7ff3b406d9d0 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.488+0000 7ff3b3400640 1 -- 172.21.15.82:0/2843319256 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a00319d0 con 0x7ff3b406d9d0 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.488+0000 7ff3b3400640 1 -- 172.21.15.82:0/2843319256 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0031d60 con 0x7ff3b406d9d0 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.489+0000 7ff3bb800640 1 -- 172.21.15.82:0/2843319256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3b406d9d0 msgr2=0x7ff3b4071550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.489+0000 7ff3bb800640 1 --2- 172.21.15.82:0/2843319256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3b406d9d0 0x7ff3b4071550 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40cd640 tx=0x7ff3a002b910 comp rx=0 tx=0).stop 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.489+0000 7ff3bb800640 1 -- 172.21.15.82:0/2843319256 shutdown_connections 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.489+0000 7ff3bb800640 1 --2- 172.21.15.82:0/2843319256 >> v2:172.21.15.90:3300/0 conn(0x7ff3b4071a90 0x7ff3b415f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.489+0000 7ff3bb800640 1 --2- 172.21.15.82:0/2843319256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3b406d9d0 0x7ff3b4071550 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.489+0000 7ff3bb800640 1 --2- 172.21.15.82:0/2843319256 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3b4144ae0 0x7ff3b406d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.489+0000 7ff3bb800640 1 -- 172.21.15.82:0/2843319256 >> 172.21.15.82:0/2843319256 conn(0x7ff3b40b1740 msgr2=0x7ff3b40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.489+0000 7ff3bb800640 1 -- 172.21.15.82:0/2843319256 shutdown_connections 2024-03-16T18:03:55.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.489+0000 7ff3bb800640 1 -- 172.21.15.82:0/2843319256 wait complete. 2024-03-16T18:03:55.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.491+0000 7ff3bb800640 1 Processor -- start 2024-03-16T18:03:55.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.491+0000 7ff3bb800640 1 -- start start 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3bb800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3b406d9d0 0x7ff3b415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3bb800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3b4071a90 0x7ff3b415f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3bb800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3b4144ae0 0x7ff3b415fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3bb800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b41f4500 con 0x7ff3b4071a90 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3bb800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b41f4920 con 0x7ff3b406d9d0 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3bb800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b41f7ec0 con 0x7ff3b4144ae0 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3b3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3b4071a90 0x7ff3b415f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3b3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3b4071a90 0x7ff3b415f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60124/0 (socket says 172.21.15.82:60124) 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3b3e00640 1 -- 172.21.15.82:0/475804260 learned_addr learned my addr 172.21.15.82:0/475804260 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3b8e00640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3b406d9d0 0x7ff3b415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3b3e00640 1 -- 172.21.15.82:0/475804260 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3b4144ae0 msgr2=0x7ff3b415fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3b3e00640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7ff3b4144ae0 0x7ff3b415fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3b3e00640 1 -- 172.21.15.82:0/475804260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3b406d9d0 msgr2=0x7ff3b415b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3b3e00640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3b406d9d0 0x7ff3b415b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.492+0000 7ff3b3e00640 1 -- 172.21.15.82:0/475804260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a00094d0 con 0x7ff3b4071a90 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.493+0000 7ff3b3e00640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3b4071a90 0x7ff3b415f940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0000c00 tx=0x7ff3a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.493+0000 7ff3b0c00640 1 -- 172.21.15.82:0/475804260 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a003abe0 con 0x7ff3b4071a90 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.493+0000 7ff3b0c00640 1 -- 172.21.15.82:0/475804260 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a003ad40 con 0x7ff3b4071a90 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.493+0000 7ff3b0c00640 1 -- 172.21.15.82:0/475804260 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0044610 con 0x7ff3b4071a90 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.493+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b41f3b60 con 0x7ff3b4071a90 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.493+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b41f4120 con 0x7ff3b4071a90 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.494+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3b413edf0 con 0x7ff3b4071a90 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.495+0000 7ff3b0c00640 1 -- 172.21.15.82:0/475804260 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7ff3a004d450 con 0x7ff3b4071a90 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.496+0000 7ff3b0c00640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff39406b3b0 0x7ff39406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.496+0000 7ff3b0c00640 1 -- 172.21.15.82:0/475804260 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff3a002d040 con 0x7ff3b4071a90 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.496+0000 7ff3b0c00640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] conn(0x7ff394071270 0x7ff394073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.496+0000 7ff3b0c00640 1 -- 172.21.15.82:0/475804260 --> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff394073d40 con 0x7ff394071270 2024-03-16T18:03:55.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.496+0000 7ff3b0c00640 1 -- 172.21.15.82:0/475804260 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3a00ae3a0 con 0x7ff3b4071a90 2024-03-16T18:03:55.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.496+0000 7ff3b8e00640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff39406b3b0 0x7ff39406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.497+0000 7ff3b9800640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] conn(0x7ff394071270 0x7ff394073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.497+0000 7ff3b8e00640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff39406b3b0 0x7ff39406d870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8006fd0 tx=0x7ff3a8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:55.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.497+0000 7ff3b9800640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] conn(0x7ff394071270 0x7ff394073690 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:55.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.498+0000 7ff3b0c00640 1 -- 172.21.15.82:0/475804260 <== osd.1 v2:172.21.15.82:6808/1061186346 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff394073d40 con 0x7ff394071270 2024-03-16T18:03:55.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.513+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 --> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff3b406e4b0 con 0x7ff394071270 2024-03-16T18:03:55.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.513+0000 7ff3b0c00640 1 -- 172.21.15.82:0/475804260 <== osd.1 v2:172.21.15.82:6808/1061186346 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff3b406e4b0 con 0x7ff394071270 2024-03-16T18:03:55.517 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T18:03:55.517 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T18:03:55.517 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.513+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] conn(0x7ff394071270 msgr2=0x7ff394073690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.513+0000 7ff3bb800640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:6808/1061186346,v1:172.21.15.82:6809/1061186346] conn(0x7ff394071270 0x7ff394073690 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.514+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff39406b3b0 msgr2=0x7ff39406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.514+0000 7ff3bb800640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7ff39406b3b0 0x7ff39406d870 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8006fd0 tx=0x7ff3a8008040 comp rx=0 tx=0).stop 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.514+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3b4071a90 msgr2=0x7ff3b415f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.514+0000 7ff3bb800640 1 --2- 172.21.15.82:0/475804260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff3b4071a90 0x7ff3b415f940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0000c00 tx=0x7ff3a00046c0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.514+0000 7ff3b9800640 1 -- 172.21.15.82:0/475804260 reap_dead start 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.514+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 shutdown_connections 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.514+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 >> 172.21.15.82:0/475804260 conn(0x7ff3b40b1740 msgr2=0x7ff3b4143a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.514+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 shutdown_connections 2024-03-16T18:03:55.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.514+0000 7ff3bb800640 1 -- 172.21.15.82:0/475804260 wait complete. 2024-03-16T18:03:55.534 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-16T18:03:55.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.728+0000 7f5998e00640 1 Processor -- start 2024-03-16T18:03:55.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.728+0000 7f5998e00640 1 -- start start 2024-03-16T18:03:55.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.730+0000 7f5998e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59941450e0 0x7f5994070940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.730+0000 7f5998e00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5994145630 0x7f599415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.730+0000 7f5998e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599406ded0 con 0x7f59940aec80 2024-03-16T18:03:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.730+0000 7f5998e00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59940af060 con 0x7f5994145630 2024-03-16T18:03:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.730+0000 7f5998e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5994072c10 con 0x7f59941450e0 2024-03-16T18:03:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.730+0000 7f5992800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5994145630 0x7f599415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.730+0000 7f5992800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5994145630 0x7f599415f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60138/0 (socket says 172.21.15.82:60138) 2024-03-16T18:03:55.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.730+0000 7f5992800640 1 -- 172.21.15.82:0/4158418102 learned_addr learned my addr 172.21.15.82:0/4158418102 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.730+0000 7f598be00640 1 --2- 172.21.15.82:0/4158418102 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59941450e0 0x7f5994070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.731+0000 7f5992800640 1 -- 172.21.15.82:0/4158418102 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59941450e0 msgr2=0x7f5994070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.731+0000 7f5992800640 1 --2- 172.21.15.82:0/4158418102 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59941450e0 0x7f5994070940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.731+0000 7f5992800640 1 -- 172.21.15.82:0/4158418102 >> v1:172.21.15.90:6789/0 conn(0x7f59940aec80 legacy=0x7f59940ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.731+0000 7f5992800640 1 -- 172.21.15.82:0/4158418102 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59940cea20 con 0x7f5994145630 2024-03-16T18:03:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.732+0000 7f5992800640 1 --2- 172.21.15.82:0/4158418102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5994145630 0x7f599415f9b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f598c009e50 tx=0x7f598c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6ea50ecc732399 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.733+0000 7f5991400640 1 -- 172.21.15.82:0/4158418102 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c02ca70 con 0x7f5994145630 2024-03-16T18:03:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.733+0000 7f5991400640 1 -- 172.21.15.82:0/4158418102 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c02cbd0 con 0x7f5994145630 2024-03-16T18:03:55.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.733+0000 7f5991400640 1 -- 172.21.15.82:0/4158418102 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c0343f0 con 0x7f5994145630 2024-03-16T18:03:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.733+0000 7f5998e00640 1 -- 172.21.15.82:0/4158418102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5994145630 msgr2=0x7f599415f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.733+0000 7f5998e00640 1 --2- 172.21.15.82:0/4158418102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5994145630 0x7f599415f9b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f598c009e50 tx=0x7f598c02bed0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.733+0000 7f5998e00640 1 -- 172.21.15.82:0/4158418102 shutdown_connections 2024-03-16T18:03:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.733+0000 7f5998e00640 1 --2- 172.21.15.82:0/4158418102 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f59941450e0 0x7f5994070940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.733+0000 7f5998e00640 1 --2- 172.21.15.82:0/4158418102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5994145630 0x7f599415f9b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.733+0000 7f5998e00640 1 -- 172.21.15.82:0/4158418102 >> 172.21.15.82:0/4158418102 conn(0x7f59940b17b0 msgr2=0x7f59940b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.734+0000 7f5998e00640 1 -- 172.21.15.82:0/4158418102 shutdown_connections 2024-03-16T18:03:55.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.734+0000 7f5998e00640 1 -- 172.21.15.82:0/4158418102 wait complete. 2024-03-16T18:03:55.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.735+0000 7f5998e00640 1 Processor -- start 2024-03-16T18:03:55.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.735+0000 7f5998e00640 1 -- start start 2024-03-16T18:03:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.736+0000 7f5998e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59940aec80 0x7f599415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.736+0000 7f5998e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59941450e0 0x7f59940b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.736+0000 7f5998e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5994145630 0x7f59940ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.736+0000 7f5998e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59940b4070 con 0x7f59941450e0 2024-03-16T18:03:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.736+0000 7f5998e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59940b31a0 con 0x7f59940aec80 2024-03-16T18:03:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.736+0000 7f5998e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59940b33f0 con 0x7f5994145630 2024-03-16T18:03:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f598be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59941450e0 0x7f59940b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f598be00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59941450e0 0x7f59940b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60142/0 (socket says 172.21.15.82:60142) 2024-03-16T18:03:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f598be00640 1 -- 172.21.15.82:0/3208622520 learned_addr learned my addr 172.21.15.82:0/3208622520 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:55.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f598be00640 1 -- 172.21.15.82:0/3208622520 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5994145630 msgr2=0x7f59940ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f598be00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5994145630 0x7f59940ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f598be00640 1 -- 172.21.15.82:0/3208622520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59940aec80 msgr2=0x7f599415b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f5991e00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59940aec80 0x7f599415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f598be00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59940aec80 0x7f599415b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f598be00640 1 -- 172.21.15.82:0/3208622520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c009b30 con 0x7f59941450e0 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f598be00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59941450e0 0x7f59940b4490 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f597c00d5f0 tx=0x7f597c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.737+0000 7f598a000640 1 -- 172.21.15.82:0/3208622520 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c0125f0 con 0x7f59941450e0 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.738+0000 7f598a000640 1 -- 172.21.15.82:0/3208622520 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c012750 con 0x7f59941450e0 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.738+0000 7f598a000640 1 -- 172.21.15.82:0/3208622520 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c012b00 con 0x7f59941450e0 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.738+0000 7f5998e00640 1 -- 172.21.15.82:0/3208622520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59940b36d0 con 0x7f59941450e0 2024-03-16T18:03:55.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.738+0000 7f5998e00640 1 -- 172.21.15.82:0/3208622520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59940b3c90 con 0x7f59941450e0 2024-03-16T18:03:55.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.739+0000 7f5998e00640 1 -- 172.21.15.82:0/3208622520 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5994073420 con 0x7f59941450e0 2024-03-16T18:03:55.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.740+0000 7f598a000640 1 -- 172.21.15.82:0/3208622520 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f597c012c60 con 0x7f59941450e0 2024-03-16T18:03:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.741+0000 7f598a000640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f597006b3b0 0x7f597006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.741+0000 7f598a000640 1 -- 172.21.15.82:0/3208622520 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f597c089e00 con 0x7f59941450e0 2024-03-16T18:03:55.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.741+0000 7f5991e00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f597006b3b0 0x7f597006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:55.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.742+0000 7f5991e00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f597006b3b0 0x7f597006d870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5984001500 tx=0x7f5984009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:55.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.745+0000 7f598a000640 1 -- 172.21.15.82:0/3208622520 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f597c054d50 con 0x7f59941450e0 2024-03-16T18:03:55.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.939+0000 7f5998e00640 1 -- 172.21.15.82:0/3208622520 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f599406d9b0 con 0x7f597006b3b0 2024-03-16T18:03:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.940+0000 7f598a000640 1 -- 172.21.15.82:0/3208622520 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5984014420 con 0x7f597006b3b0 2024-03-16T18:03:55.944 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 1 to deep-scrub 2024-03-16T18:03:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.943+0000 7f5998e00640 1 -- 172.21.15.82:0/3208622520 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f597006b3b0 msgr2=0x7f597006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.943+0000 7f5998e00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f597006b3b0 0x7f597006d870 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5984001500 tx=0x7f5984009040 comp rx=0 tx=0).stop 2024-03-16T18:03:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.943+0000 7f5998e00640 1 -- 172.21.15.82:0/3208622520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59941450e0 msgr2=0x7f59940b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.943+0000 7f5998e00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59941450e0 0x7f59940b4490 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f597c00d5f0 tx=0x7f597c00dab0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.943+0000 7f5998e00640 1 -- 172.21.15.82:0/3208622520 shutdown_connections 2024-03-16T18:03:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.943+0000 7f5998e00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f597006b3b0 0x7f597006d870 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.943+0000 7f5998e00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5994145630 0x7f59940ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.943+0000 7f5998e00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59941450e0 0x7f59940b4490 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.943+0000 7f5998e00640 1 --2- 172.21.15.82:0/3208622520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59940aec80 0x7f599415b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.943+0000 7f5998e00640 1 -- 172.21.15.82:0/3208622520 >> 172.21.15.82:0/3208622520 conn(0x7f59940b17b0 msgr2=0x7f59940cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.944+0000 7f5998e00640 1 -- 172.21.15.82:0/3208622520 shutdown_connections 2024-03-16T18:03:55.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:55.944+0000 7f5998e00640 1 -- 172.21.15.82:0/3208622520 wait complete. 2024-03-16T18:03:55.967 INFO:tasks.ceph:Scrubbing osd.2 2024-03-16T18:03:55.967 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-16T18:03:56.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.176+0000 7f37c4000640 1 Processor -- start 2024-03-16T18:03:56.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.176+0000 7f37c4000640 1 -- start start 2024-03-16T18:03:56.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.178+0000 7f37c4000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f37bc15e410 0x7f37bc0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.178+0000 7f37c4000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37bc0d2390 0x7f37bc0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.178+0000 7f37c4000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37bc0cf770 con 0x7f37bc15e030 2024-03-16T18:03:56.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.178+0000 7f37c4000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37bc0d2d60 con 0x7f37bc0d2390 2024-03-16T18:03:56.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.178+0000 7f37c4000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0ce8c0 con 0x7f37bc15e410 2024-03-16T18:03:56.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.178+0000 7f37c2000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37bc0d2390 0x7f37bc0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.178+0000 7f37c2000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37bc0d2390 0x7f37bc0d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60156/0 (socket says 172.21.15.82:60156) 2024-03-16T18:03:56.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.178+0000 7f37c2000640 1 -- 172.21.15.82:0/1613976195 learned_addr learned my addr 172.21.15.82:0/1613976195 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:56.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.179+0000 7f37c0c00640 1 --2- 172.21.15.82:0/1613976195 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f37bc15e410 0x7f37bc0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.179+0000 7f37c2000640 1 -- 172.21.15.82:0/1613976195 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f37bc15e410 msgr2=0x7f37bc0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:56.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.179+0000 7f37c2000640 1 --2- 172.21.15.82:0/1613976195 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f37bc15e410 0x7f37bc0cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.179+0000 7f37c2000640 1 -- 172.21.15.82:0/1613976195 >> v1:172.21.15.90:6789/0 conn(0x7f37bc15e030 legacy=0x7f37bc0d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.179+0000 7f37c2000640 1 -- 172.21.15.82:0/1613976195 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc0ceb60 con 0x7f37bc0d2390 2024-03-16T18:03:56.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.180+0000 7f37c2000640 1 --2- 172.21.15.82:0/1613976195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37bc0d2390 0x7f37bc0d2820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009e50 tx=0x7f37b402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568774c053ea72b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.180+0000 7f37bbe00640 1 -- 172.21.15.82:0/1613976195 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b402cb00 con 0x7f37bc0d2390 2024-03-16T18:03:56.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.180+0000 7f37bbe00640 1 -- 172.21.15.82:0/1613976195 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b402cc60 con 0x7f37bc0d2390 2024-03-16T18:03:56.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.180+0000 7f37bbe00640 1 -- 172.21.15.82:0/1613976195 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b40343f0 con 0x7f37bc0d2390 2024-03-16T18:03:56.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.181+0000 7f37c4000640 1 -- 172.21.15.82:0/1613976195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37bc0d2390 msgr2=0x7f37bc0d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.181+0000 7f37c4000640 1 --2- 172.21.15.82:0/1613976195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37bc0d2390 0x7f37bc0d2820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f37b4009e50 tx=0x7f37b402bf60 comp rx=0 tx=0).stop 2024-03-16T18:03:56.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.181+0000 7f37c4000640 1 -- 172.21.15.82:0/1613976195 shutdown_connections 2024-03-16T18:03:56.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.181+0000 7f37c4000640 1 --2- 172.21.15.82:0/1613976195 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f37bc15e410 0x7f37bc0cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.181+0000 7f37c4000640 1 --2- 172.21.15.82:0/1613976195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37bc0d2390 0x7f37bc0d2820 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.181+0000 7f37c4000640 1 -- 172.21.15.82:0/1613976195 >> 172.21.15.82:0/1613976195 conn(0x7f37bc0d3db0 msgr2=0x7f37bc0d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:56.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.181+0000 7f37c4000640 1 -- 172.21.15.82:0/1613976195 shutdown_connections 2024-03-16T18:03:56.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.181+0000 7f37c4000640 1 -- 172.21.15.82:0/1613976195 wait complete. 2024-03-16T18:03:56.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.183+0000 7f37c4000640 1 Processor -- start 2024-03-16T18:03:56.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.183+0000 7f37c4000640 1 -- start start 2024-03-16T18:03:56.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c4000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37bc0d2390 0x7f37bc0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c4000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37bc15e030 0x7f37bc0b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c4000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f37bc15e410 0x7f37bc06c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c4000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0ccaf0 con 0x7f37bc15e030 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c4000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0ccdd0 con 0x7f37bc0d2390 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c4000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc0cd0b0 con 0x7f37bc15e410 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c0c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37bc15e030 0x7f37bc0b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c0c00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37bc15e030 0x7f37bc0b6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60166/0 (socket says 172.21.15.82:60166) 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c0c00640 1 -- 172.21.15.82:0/1082350977 learned_addr learned my addr 172.21.15.82:0/1082350977 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c1600640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37bc0d2390 0x7f37bc0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c0c00640 1 -- 172.21.15.82:0/1082350977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f37bc15e410 msgr2=0x7f37bc06c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c2000640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f37bc15e410 0x7f37bc06c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c0c00640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f37bc15e410 0x7f37bc06c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c0c00640 1 -- 172.21.15.82:0/1082350977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37bc0d2390 msgr2=0x7f37bc0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c0c00640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37bc0d2390 0x7f37bc0cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.184+0000 7f37c0c00640 1 -- 172.21.15.82:0/1082350977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b4009b30 con 0x7f37bc15e030 2024-03-16T18:03:56.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.185+0000 7f37c1600640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37bc0d2390 0x7f37bc0cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:56.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.185+0000 7f37c0c00640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37bc15e030 0x7f37bc0b6b40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f37a800d800 tx=0x7f37a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.185+0000 7f37b9600640 1 -- 172.21.15.82:0/1082350977 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a800b6c0 con 0x7f37bc15e030 2024-03-16T18:03:56.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.185+0000 7f37b9600640 1 -- 172.21.15.82:0/1082350977 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a800b820 con 0x7f37bc15e030 2024-03-16T18:03:56.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.185+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc0cd390 con 0x7f37bc15e030 2024-03-16T18:03:56.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.185+0000 7f37b9600640 1 -- 172.21.15.82:0/1082350977 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a800bbb0 con 0x7f37bc15e030 2024-03-16T18:03:56.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.185+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc0b7080 con 0x7f37bc15e030 2024-03-16T18:03:56.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.186+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f37bc0d3540 con 0x7f37bc15e030 2024-03-16T18:03:56.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.187+0000 7f37b9600640 1 -- 172.21.15.82:0/1082350977 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f37a800bd10 con 0x7f37bc15e030 2024-03-16T18:03:56.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.188+0000 7f37b9600640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f379c06b3b0 0x7f379c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.188+0000 7f37b9600640 1 -- 172.21.15.82:0/1082350977 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f37a8089f40 con 0x7f37bc15e030 2024-03-16T18:03:56.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.188+0000 7f37c1600640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f379c06b3b0 0x7f379c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.188+0000 7f37b9600640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] conn(0x7f379c071270 0x7f379c073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.188+0000 7f37b9600640 1 -- 172.21.15.82:0/1082350977 --> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f379c073d40 con 0x7f379c071270 2024-03-16T18:03:56.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.188+0000 7f37b9600640 1 -- 172.21.15.82:0/1082350977 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37a808c520 con 0x7f37bc15e030 2024-03-16T18:03:56.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.189+0000 7f37c2000640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] conn(0x7f379c071270 0x7f379c073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.189+0000 7f37c1600640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f379c06b3b0 0x7f379c06d870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f37ac001680 tx=0x7f37ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.189+0000 7f37c2000640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] conn(0x7f379c071270 0x7f379c073690 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.190+0000 7f37b9600640 1 -- 172.21.15.82:0/1082350977 <== osd.2 v2:172.21.15.82:6824/1740405378 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f379c073d40 con 0x7f379c071270 2024-03-16T18:03:56.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.204+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 --> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f37bc0cf400 con 0x7f379c071270 2024-03-16T18:03:56.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.205+0000 7f37b9600640 1 -- 172.21.15.82:0/1082350977 <== osd.2 v2:172.21.15.82:6824/1740405378 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f37b402ca00 con 0x7f379c071270 2024-03-16T18:03:56.209 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T18:03:56.209 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T18:03:56.209 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.205+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] conn(0x7f379c071270 msgr2=0x7f379c073690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.206+0000 7f37c4000640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:6824/1740405378,v1:172.21.15.82:6825/1740405378] conn(0x7f379c071270 0x7f379c073690 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.206+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f379c06b3b0 msgr2=0x7f379c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.206+0000 7f37c4000640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f379c06b3b0 0x7f379c06d870 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f37ac001680 tx=0x7f37ac009040 comp rx=0 tx=0).stop 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.206+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37bc15e030 msgr2=0x7f37bc0b6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.206+0000 7f37c4000640 1 --2- 172.21.15.82:0/1082350977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37bc15e030 0x7f37bc0b6b40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f37a800d800 tx=0x7f37a800dcc0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.206+0000 7f37c2000640 1 -- 172.21.15.82:0/1082350977 reap_dead start 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.206+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 shutdown_connections 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.206+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 >> 172.21.15.82:0/1082350977 conn(0x7f37bc0d3db0 msgr2=0x7f37bc0c9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.206+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 shutdown_connections 2024-03-16T18:03:56.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.206+0000 7f37c4000640 1 -- 172.21.15.82:0/1082350977 wait complete. 2024-03-16T18:03:56.228 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-16T18:03:56.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.423+0000 7f78fd200640 1 Processor -- start 2024-03-16T18:03:56.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.423+0000 7f78fd200640 1 -- start start 2024-03-16T18:03:56.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.425+0000 7f78fd200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f81451e0 0x7f78f8145b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.425+0000 7f78fd200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f78f80b31c0 0x7f78f8147a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.425+0000 7f78fd200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f8148820 con 0x7f78f81455c0 2024-03-16T18:03:56.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.425+0000 7f78fd200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f8148b00 con 0x7f78f80b31c0 2024-03-16T18:03:56.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.425+0000 7f78fd200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f80cdaa0 con 0x7f78f81451e0 2024-03-16T18:03:56.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.426+0000 7f78f6800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f78f80b31c0 0x7f78f8147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.426+0000 7f78f5e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f81451e0 0x7f78f8145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.426+0000 7f78f6800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f78f80b31c0 0x7f78f8147a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:54942/0 (socket says 172.21.15.82:54942) 2024-03-16T18:03:56.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.426+0000 7f78f6800640 1 -- 172.21.15.82:0/3697804957 learned_addr learned my addr 172.21.15.82:0/3697804957 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:56.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.426+0000 7f78f6800640 1 -- 172.21.15.82:0/3697804957 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f81451e0 msgr2=0x7f78f8145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:56.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.426+0000 7f78f6800640 1 --2- 172.21.15.82:0/3697804957 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f81451e0 0x7f78f8145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.426+0000 7f78f6800640 1 -- 172.21.15.82:0/3697804957 >> v1:172.21.15.90:6789/0 conn(0x7f78f81455c0 legacy=0x7f78f80b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.426+0000 7f78f6800640 1 -- 172.21.15.82:0/3697804957 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f80cdd80 con 0x7f78f80b31c0 2024-03-16T18:03:56.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.426+0000 7f78f5400640 1 -- 172.21.15.82:0/3697804957 <== mon.1 v1:172.21.15.90:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643940875 0 0) 0x7f78f8148820 con 0x7f78f81455c0 2024-03-16T18:03:56.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.428+0000 7f78f6800640 1 --2- 172.21.15.82:0/3697804957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f80b31c0 0x7f78f8147a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009e50 tx=0x7f78f002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=336c14562ba69ae7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.428+0000 7f78f5400640 1 -- 172.21.15.82:0/3697804957 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f002cb00 con 0x7f78f80b31c0 2024-03-16T18:03:56.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.428+0000 7f78f5400640 1 -- 172.21.15.82:0/3697804957 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f002cc60 con 0x7f78f80b31c0 2024-03-16T18:03:56.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.428+0000 7f78f5400640 1 -- 172.21.15.82:0/3697804957 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f00343f0 con 0x7f78f80b31c0 2024-03-16T18:03:56.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.429+0000 7f78fd200640 1 -- 172.21.15.82:0/3697804957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f80b31c0 msgr2=0x7f78f8147a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.429+0000 7f78fd200640 1 --2- 172.21.15.82:0/3697804957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f80b31c0 0x7f78f8147a90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f78f0009e50 tx=0x7f78f002bf60 comp rx=0 tx=0).stop 2024-03-16T18:03:56.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.429+0000 7f78fd200640 1 -- 172.21.15.82:0/3697804957 shutdown_connections 2024-03-16T18:03:56.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.429+0000 7f78fd200640 1 --2- 172.21.15.82:0/3697804957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f80b31c0 0x7f78f8147a90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.429+0000 7f78fd200640 1 --2- 172.21.15.82:0/3697804957 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f81451e0 0x7f78f8145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.429+0000 7f78fd200640 1 -- 172.21.15.82:0/3697804957 >> 172.21.15.82:0/3697804957 conn(0x7f78f80b17b0 msgr2=0x7f78f80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:56.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.429+0000 7f78fd200640 1 -- 172.21.15.82:0/3697804957 shutdown_connections 2024-03-16T18:03:56.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.429+0000 7f78fd200640 1 -- 172.21.15.82:0/3697804957 wait complete. 2024-03-16T18:03:56.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.431+0000 7f78fd200640 1 Processor -- start 2024-03-16T18:03:56.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.431+0000 7f78fd200640 1 -- start start 2024-03-16T18:03:56.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.432+0000 7f78fd200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f80b31c0 0x7f78f81421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.432+0000 7f78fd200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f81451e0 0x7f78f81426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.432+0000 7f78fd200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f81455c0 0x7f78f81fb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.432+0000 7f78fd200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f8148c40 con 0x7f78f81451e0 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.432+0000 7f78fd200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f814b920 con 0x7f78f81455c0 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.432+0000 7f78fd200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f8148e90 con 0x7f78f80b31c0 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.432+0000 7f78efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f81451e0 0x7f78f81426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.432+0000 7f78efe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f81451e0 0x7f78f81426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60174/0 (socket says 172.21.15.82:60174) 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.432+0000 7f78efe00640 1 -- 172.21.15.82:0/78830581 learned_addr learned my addr 172.21.15.82:0/78830581 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.432+0000 7f78f6800640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f81455c0 0x7f78f81fb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78f5e00640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f80b31c0 0x7f78f81421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78efe00640 1 -- 172.21.15.82:0/78830581 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f80b31c0 msgr2=0x7f78f81421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78efe00640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f80b31c0 0x7f78f81421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78efe00640 1 -- 172.21.15.82:0/78830581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f81455c0 msgr2=0x7f78f81fb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78efe00640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f81455c0 0x7f78f81fb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78efe00640 1 -- 172.21.15.82:0/78830581 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f0009b30 con 0x7f78f81451e0 2024-03-16T18:03:56.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78efe00640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f81451e0 0x7f78f81426f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f78e000c3e0 tx=0x7f78e000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78ee000640 1 -- 172.21.15.82:0/78830581 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00035f0 con 0x7f78f81451e0 2024-03-16T18:03:56.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78ee000640 1 -- 172.21.15.82:0/78830581 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e0003750 con 0x7f78f81451e0 2024-03-16T18:03:56.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78ee000640 1 -- 172.21.15.82:0/78830581 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e0003b20 con 0x7f78f81451e0 2024-03-16T18:03:56.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.433+0000 7f78fd200640 1 -- 172.21.15.82:0/78830581 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f8149170 con 0x7f78f81451e0 2024-03-16T18:03:56.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.434+0000 7f78fd200640 1 -- 172.21.15.82:0/78830581 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f8162e60 con 0x7f78f81451e0 2024-03-16T18:03:56.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.435+0000 7f78fd200640 1 -- 172.21.15.82:0/78830581 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f8162ff0 con 0x7f78f81451e0 2024-03-16T18:03:56.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.435+0000 7f78ee000640 1 -- 172.21.15.82:0/78830581 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f78e0003c80 con 0x7f78f81451e0 2024-03-16T18:03:56.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.436+0000 7f78ee000640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78d406b3b0 0x7f78d406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.436+0000 7f78ee000640 1 -- 172.21.15.82:0/78830581 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f78e0088f20 con 0x7f78f81451e0 2024-03-16T18:03:56.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.437+0000 7f78f5e00640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78d406b3b0 0x7f78d406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.437+0000 7f78f5e00640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78d406b3b0 0x7f78d406d870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009ad0 tx=0x7f78e8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.441+0000 7f78ee000640 1 -- 172.21.15.82:0/78830581 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f78e0053e80 con 0x7f78f81451e0 2024-03-16T18:03:56.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.635+0000 7f78fd200640 1 -- 172.21.15.82:0/78830581 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f78f8149690 con 0x7f78d406b3b0 2024-03-16T18:03:56.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.636+0000 7f78ee000640 1 -- 172.21.15.82:0/78830581 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f78f8149690 con 0x7f78d406b3b0 2024-03-16T18:03:56.640 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 2 to deep-scrub 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 -- 172.21.15.82:0/78830581 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78d406b3b0 msgr2=0x7f78d406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78d406b3b0 0x7f78d406d870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009ad0 tx=0x7f78e8008040 comp rx=0 tx=0).stop 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 -- 172.21.15.82:0/78830581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f81451e0 msgr2=0x7f78f81426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f81451e0 0x7f78f81426f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f78e000c3e0 tx=0x7f78e000c8a0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 -- 172.21.15.82:0/78830581 shutdown_connections 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f78d406b3b0 0x7f78d406d870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78f81455c0 0x7f78f81fb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78f81451e0 0x7f78f81426f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 --2- 172.21.15.82:0/78830581 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f78f80b31c0 0x7f78f81421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 -- 172.21.15.82:0/78830581 >> 172.21.15.82:0/78830581 conn(0x7f78f80b17b0 msgr2=0x7f78f8143560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:56.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 -- 172.21.15.82:0/78830581 shutdown_connections 2024-03-16T18:03:56.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.639+0000 7f78fd200640 1 -- 172.21.15.82:0/78830581 wait complete. 2024-03-16T18:03:56.662 INFO:tasks.ceph:Scrubbing osd.3 2024-03-16T18:03:56.662 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-16T18:03:56.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.869+0000 7f5366000640 1 Processor -- start 2024-03-16T18:03:56.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.869+0000 7f5366000640 1 -- start start 2024-03-16T18:03:56.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.871+0000 7f5366000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600cf140 0x7f5360071c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.871+0000 7f5366000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53600cdca0 con 0x7f536006dba0 2024-03-16T18:03:56.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.871+0000 7f5366000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5360144ec0 con 0x7f5360144ae0 2024-03-16T18:03:56.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.871+0000 7f5366000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53600722d0 con 0x7f53600cf140 2024-03-16T18:03:56.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.871+0000 7f535ea00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f536006dba0 0x7f5360071550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58996/0 (socket says 172.21.15.82:58996) 2024-03-16T18:03:56.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.871+0000 7f535ea00640 1 -- 172.21.15.82:0/2721134456 learned_addr learned my addr 172.21.15.82:0/2721134456 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:56.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535fe00640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600cf140 0x7f5360071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029838065 0 0) 0x7f53600cdca0 con 0x7f536006dba0 2024-03-16T18:03:56.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5344003520 con 0x7f536006dba0 2024-03-16T18:03:56.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3423228005 0 0) 0x7f5344003520 con 0x7f536006dba0 2024-03-16T18:03:56.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600cf140 msgr2=0x7f5360071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:56.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600cf140 0x7f5360071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 >> v1:172.21.15.90:6789/0 conn(0x7f5360144ae0 legacy=0x7f536006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53600725b0 con 0x7f536006dba0 2024-03-16T18:03:56.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f534c001670 con 0x7f536006dba0 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 >> v1:172.21.15.82:6789/0 conn(0x7f536006dba0 legacy=0x7f5360071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53440039a0 0x7f5344003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 0x7f5344004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600cf140 0x7f53440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53600722d0 con 0x7f53440047c0 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534400bab0 con 0x7f53440039a0 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.872+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534400bc70 con 0x7f53600cf140 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.873+0000 7f535ea00640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 0x7f5344004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.873+0000 7f535f400640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53440039a0 0x7f5344003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.873+0000 7f535ea00640 1 -- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600cf140 msgr2=0x7f53440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.873+0000 7f535ea00640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600cf140 0x7f53440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.873+0000 7f535ea00640 1 -- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53440039a0 msgr2=0x7f5344003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:56.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.873+0000 7f535ea00640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53440039a0 0x7f5344003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.873+0000 7f535ea00640 1 -- 172.21.15.82:0/2721134456 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344003520 con 0x7f53440047c0 2024-03-16T18:03:56.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.873+0000 7f535f400640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53440039a0 0x7f5344003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.874+0000 7f535ea00640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 0x7f5344004c40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f534c00c240 tx=0x7f53440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9640f16faa54800b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.875+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c005ad0 con 0x7f53440047c0 2024-03-16T18:03:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.875+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c005c30 con 0x7f53440047c0 2024-03-16T18:03:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.875+0000 7f535e000640 1 -- 172.21.15.82:0/2721134456 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c02c8b0 con 0x7f53440047c0 2024-03-16T18:03:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.875+0000 7f5366000640 1 -- 172.21.15.82:0/2721134456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 msgr2=0x7f5344004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.875+0000 7f5366000640 1 --2- 172.21.15.82:0/2721134456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 0x7f5344004c40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f534c00c240 tx=0x7f53440057f0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.875+0000 7f535fe00640 1 -- 172.21.15.82:0/2721134456 reap_dead start 2024-03-16T18:03:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.875+0000 7f5366000640 1 -- 172.21.15.82:0/2721134456 shutdown_connections 2024-03-16T18:03:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.875+0000 7f5366000640 1 -- 172.21.15.82:0/2721134456 >> 172.21.15.82:0/2721134456 conn(0x7f53600b1740 msgr2=0x7f53600b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.875+0000 7f5366000640 1 -- 172.21.15.82:0/2721134456 shutdown_connections 2024-03-16T18:03:56.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.875+0000 7f5366000640 1 -- 172.21.15.82:0/2721134456 wait complete. 2024-03-16T18:03:56.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.877+0000 7f5366000640 1 Processor -- start 2024-03-16T18:03:56.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.877+0000 7f5366000640 1 -- start start 2024-03-16T18:03:56.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.878+0000 7f5366000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 0x7f536015d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.878+0000 7f5366000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600ad850 0x7f536013d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.878+0000 7f5366000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53600adcd0 0x7f536013d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.878+0000 7f5366000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536015d560 con 0x7f53440047c0 2024-03-16T18:03:56.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.878+0000 7f5366000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536013c9d0 con 0x7f53600adcd0 2024-03-16T18:03:56.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.878+0000 7f5366000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f536015c8e0 con 0x7f53600ad850 2024-03-16T18:03:56.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.878+0000 7f535f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 0x7f536015d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.878+0000 7f535f400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 0x7f536015d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60184/0 (socket says 172.21.15.82:60184) 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.878+0000 7f535f400640 1 -- 172.21.15.82:0/2255347300 learned_addr learned my addr 172.21.15.82:0/2255347300 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f535fe00640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53600adcd0 0x7f536013d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f535ea00640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600ad850 0x7f536013d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f535f400640 1 -- 172.21.15.82:0/2255347300 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600ad850 msgr2=0x7f536013d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f535f400640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f53600ad850 0x7f536013d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f535f400640 1 -- 172.21.15.82:0/2255347300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53600adcd0 msgr2=0x7f536013d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f535f400640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53600adcd0 0x7f536013d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f535f400640 1 -- 172.21.15.82:0/2255347300 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536015cbc0 con 0x7f53440047c0 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f535f400640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 0x7f536015d980 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f535000d7a0 tx=0x7f535000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f5357400640 1 -- 172.21.15.82:0/2255347300 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350005270 con 0x7f53440047c0 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f5357400640 1 -- 172.21.15.82:0/2255347300 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53500053d0 con 0x7f53440047c0 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536015cea0 con 0x7f53440047c0 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.879+0000 7f5357400640 1 -- 172.21.15.82:0/2255347300 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535000b8f0 con 0x7f53440047c0 2024-03-16T18:03:56.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.880+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536015c4b0 con 0x7f53440047c0 2024-03-16T18:03:56.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.880+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5360142b40 con 0x7f53440047c0 2024-03-16T18:03:56.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.882+0000 7f5357400640 1 -- 172.21.15.82:0/2255347300 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f535000ba50 con 0x7f53440047c0 2024-03-16T18:03:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.882+0000 7f5357400640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f534006b3e0 0x7f534006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.883+0000 7f5357400640 1 -- 172.21.15.82:0/2255347300 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5350089780 con 0x7f53440047c0 2024-03-16T18:03:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.883+0000 7f535ea00640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f534006b3e0 0x7f534006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.883+0000 7f5357400640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] conn(0x7f53400712a0 0x7f53400736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.883+0000 7f5357400640 1 -- 172.21.15.82:0/2255347300 --> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5340073d70 con 0x7f53400712a0 2024-03-16T18:03:56.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.883+0000 7f5357400640 1 -- 172.21.15.82:0/2255347300 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f535008be40 con 0x7f53440047c0 2024-03-16T18:03:56.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.883+0000 7f535fe00640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] conn(0x7f53400712a0 0x7f53400736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:56.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.883+0000 7f535ea00640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f534006b3e0 0x7f534006d8a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f534c034000 tx=0x7f534c009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.884+0000 7f535fe00640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] conn(0x7f53400712a0 0x7f53400736c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:56.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.884+0000 7f5357400640 1 -- 172.21.15.82:0/2255347300 <== osd.3 v2:172.21.15.82:6816/4087215098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5340073d70 con 0x7f53400712a0 2024-03-16T18:03:56.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.899+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 --> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f53600cd930 con 0x7f53400712a0 2024-03-16T18:03:56.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.899+0000 7f5357400640 1 -- 172.21.15.82:0/2255347300 <== osd.3 v2:172.21.15.82:6816/4087215098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5358011040 con 0x7f53400712a0 2024-03-16T18:03:56.903 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T18:03:56.903 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T18:03:56.903 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T18:03:56.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.900+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] conn(0x7f53400712a0 msgr2=0x7f53400736c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.900+0000 7f5366000640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:6816/4087215098,v1:172.21.15.82:6817/4087215098] conn(0x7f53400712a0 0x7f53400736c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:56.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.900+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f534006b3e0 msgr2=0x7f534006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.900+0000 7f5366000640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f534006b3e0 0x7f534006d8a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f534c034000 tx=0x7f534c009030 comp rx=0 tx=0).stop 2024-03-16T18:03:56.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.900+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 msgr2=0x7f536015d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:56.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.900+0000 7f5366000640 1 --2- 172.21.15.82:0/2255347300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53440047c0 0x7f536015d980 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f535000d7a0 tx=0x7f535000dc60 comp rx=0 tx=0).stop 2024-03-16T18:03:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.900+0000 7f535fe00640 1 -- 172.21.15.82:0/2255347300 reap_dead start 2024-03-16T18:03:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.900+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 shutdown_connections 2024-03-16T18:03:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.900+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 >> 172.21.15.82:0/2255347300 conn(0x7f53600b1740 msgr2=0x7f5360071550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.901+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 shutdown_connections 2024-03-16T18:03:56.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:56.901+0000 7f5366000640 1 -- 172.21.15.82:0/2255347300 wait complete. 2024-03-16T18:03:56.922 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-16T18:03:57.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.123+0000 7f9f1e200640 1 Processor -- start 2024-03-16T18:03:57.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.123+0000 7f9f1e200640 1 -- start start 2024-03-16T18:03:57.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f1e200640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9f180b2e50 0x7f9f180b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f1e200640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f9f180b3fa0 0x7f9f1806c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f1e200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f180b43f0 con 0x7f9f1815ae00 2024-03-16T18:03:57.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f1e200640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f18072ca0 con 0x7f9f180b2e50 2024-03-16T18:03:57.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f1e200640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f180b32b0 con 0x7f9f180b3fa0 2024-03-16T18:03:57.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f0fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9f180b2e50 0x7f9f180b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f0fe00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9f180b2e50 0x7f9f180b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60190/0 (socket says 172.21.15.82:60190) 2024-03-16T18:03:57.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f17400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9f1815ae00 0x7f9f18072590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59002/0 (socket says 172.21.15.82:59002) 2024-03-16T18:03:57.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f0fe00640 1 -- 172.21.15.82:0/1110518966 learned_addr learned my addr 172.21.15.82:0/1110518966 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:57.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f17e00640 1 --2- 172.21.15.82:0/1110518966 >> v2:172.21.15.90:3300/0 conn(0x7f9f180b3fa0 0x7f9f1806c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f0fe00640 1 -- 172.21.15.82:0/1110518966 >> v2:172.21.15.90:3300/0 conn(0x7f9f180b3fa0 msgr2=0x7f9f1806c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:57.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f0fe00640 1 --2- 172.21.15.82:0/1110518966 >> v2:172.21.15.90:3300/0 conn(0x7f9f180b3fa0 0x7f9f1806c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f0fe00640 1 -- 172.21.15.82:0/1110518966 >> v1:172.21.15.82:6789/0 conn(0x7f9f1815ae00 legacy=0x7f9f18072590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.125+0000 7f9f0fe00640 1 -- 172.21.15.82:0/1110518966 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f180cf610 con 0x7f9f180b2e50 2024-03-16T18:03:57.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.127+0000 7f9f0fe00640 1 --2- 172.21.15.82:0/1110518966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b2e50 0x7f9f180b3a60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9f18073290 tx=0x7f9f0002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51580f669357412a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:57.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.127+0000 7f9f16a00640 1 -- 172.21.15.82:0/1110518966 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f00031870 con 0x7f9f180b2e50 2024-03-16T18:03:57.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.127+0000 7f9f16a00640 1 -- 172.21.15.82:0/1110518966 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f000319d0 con 0x7f9f180b2e50 2024-03-16T18:03:57.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.127+0000 7f9f16a00640 1 -- 172.21.15.82:0/1110518966 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f00031d60 con 0x7f9f180b2e50 2024-03-16T18:03:57.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.127+0000 7f9f1e200640 1 -- 172.21.15.82:0/1110518966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b2e50 msgr2=0x7f9f180b3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.127+0000 7f9f1e200640 1 --2- 172.21.15.82:0/1110518966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b2e50 0x7f9f180b3a60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9f18073290 tx=0x7f9f0002b910 comp rx=0 tx=0).stop 2024-03-16T18:03:57.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.128+0000 7f9f1e200640 1 -- 172.21.15.82:0/1110518966 shutdown_connections 2024-03-16T18:03:57.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.128+0000 7f9f1e200640 1 --2- 172.21.15.82:0/1110518966 >> v2:172.21.15.90:3300/0 conn(0x7f9f180b3fa0 0x7f9f1806c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.128+0000 7f9f1e200640 1 --2- 172.21.15.82:0/1110518966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b2e50 0x7f9f180b3a60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.128+0000 7f9f1e200640 1 -- 172.21.15.82:0/1110518966 >> 172.21.15.82:0/1110518966 conn(0x7f9f180b17b0 msgr2=0x7f9f180b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:57.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.128+0000 7f9f1e200640 1 -- 172.21.15.82:0/1110518966 shutdown_connections 2024-03-16T18:03:57.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.128+0000 7f9f1e200640 1 -- 172.21.15.82:0/1110518966 wait complete. 2024-03-16T18:03:57.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.129+0000 7f9f1e200640 1 Processor -- start 2024-03-16T18:03:57.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.129+0000 7f9f1e200640 1 -- start start 2024-03-16T18:03:57.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.130+0000 7f9f1e200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180b2e50 0x7f9f1815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.130+0000 7f9f1e200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b3fa0 0x7f9f180b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.130+0000 7f9f1e200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f1815ae00 0x7f9f1814d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.130+0000 7f9f1e200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1814f6e0 con 0x7f9f180b3fa0 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.130+0000 7f9f1e200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f18150180 con 0x7f9f180b2e50 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.130+0000 7f9f1e200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1814fb70 con 0x7f9f1815ae00 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f0fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b3fa0 0x7f9f180b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f0fe00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b3fa0 0x7f9f180b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60198/0 (socket says 172.21.15.82:60198) 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f0fe00640 1 -- 172.21.15.82:0/3381198447 learned_addr learned my addr 172.21.15.82:0/3381198447 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f17400640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180b2e50 0x7f9f1815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f17e00640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f1815ae00 0x7f9f1814d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f0fe00640 1 -- 172.21.15.82:0/3381198447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f1815ae00 msgr2=0x7f9f1814d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f0fe00640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f1815ae00 0x7f9f1814d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f0fe00640 1 -- 172.21.15.82:0/3381198447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180b2e50 msgr2=0x7f9f1815a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f0fe00640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180b2e50 0x7f9f1815a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f0fe00640 1 -- 172.21.15.82:0/3381198447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f000094d0 con 0x7f9f180b3fa0 2024-03-16T18:03:57.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f17400640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180b2e50 0x7f9f1815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:57.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.131+0000 7f9f0fe00640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b3fa0 0x7f9f180b0130 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9f0002b950 tx=0x7f9f000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:57.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.132+0000 7f9f0f400640 1 -- 172.21.15.82:0/3381198447 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0003ad30 con 0x7f9f180b3fa0 2024-03-16T18:03:57.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.132+0000 7f9f0f400640 1 -- 172.21.15.82:0/3381198447 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f000443f0 con 0x7f9f180b3fa0 2024-03-16T18:03:57.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.132+0000 7f9f0f400640 1 -- 172.21.15.82:0/3381198447 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f000447a0 con 0x7f9f180b3fa0 2024-03-16T18:03:57.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.132+0000 7f9f1e200640 1 -- 172.21.15.82:0/3381198447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1814f400 con 0x7f9f180b3fa0 2024-03-16T18:03:57.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.132+0000 7f9f1e200640 1 -- 172.21.15.82:0/3381198447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1814f040 con 0x7f9f180b3fa0 2024-03-16T18:03:57.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.133+0000 7f9f1e200640 1 -- 172.21.15.82:0/3381198447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1814f9d0 con 0x7f9f180b3fa0 2024-03-16T18:03:57.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.134+0000 7f9f0f400640 1 -- 172.21.15.82:0/3381198447 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f9f00044900 con 0x7f9f180b3fa0 2024-03-16T18:03:57.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.135+0000 7f9f0f400640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9ef406b3b0 0x7f9ef406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.135+0000 7f9f0f400640 1 -- 172.21.15.82:0/3381198447 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9f0002d040 con 0x7f9f180b3fa0 2024-03-16T18:03:57.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.135+0000 7f9f17400640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9ef406b3b0 0x7f9ef406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.136+0000 7f9f17400640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9ef406b3b0 0x7f9ef406d870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9f08000f60 tx=0x7f9f08009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:57.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.140+0000 7f9f0f400640 1 -- 172.21.15.82:0/3381198447 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9f1814f9d0 con 0x7f9f180b3fa0 2024-03-16T18:03:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.334+0000 7f9f1e200640 1 -- 172.21.15.82:0/3381198447 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f180cdd80 con 0x7f9ef406b3b0 2024-03-16T18:03:57.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.335+0000 7f9f0f400640 1 -- 172.21.15.82:0/3381198447 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9f080070d0 con 0x7f9ef406b3b0 2024-03-16T18:03:57.338 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 3 to deep-scrub 2024-03-16T18:03:57.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.337+0000 7f9f1e200640 1 -- 172.21.15.82:0/3381198447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9ef406b3b0 msgr2=0x7f9ef406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.337+0000 7f9f1e200640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9ef406b3b0 0x7f9ef406d870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9f08000f60 tx=0x7f9f08009040 comp rx=0 tx=0).stop 2024-03-16T18:03:57.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.337+0000 7f9f1e200640 1 -- 172.21.15.82:0/3381198447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b3fa0 msgr2=0x7f9f180b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.337+0000 7f9f1e200640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b3fa0 0x7f9f180b0130 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9f0002b950 tx=0x7f9f000046c0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.337+0000 7f9f1e200640 1 -- 172.21.15.82:0/3381198447 shutdown_connections 2024-03-16T18:03:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.337+0000 7f9f1e200640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f9ef406b3b0 0x7f9ef406d870 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.337+0000 7f9f1e200640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f9f1815ae00 0x7f9f1814d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.337+0000 7f9f1e200640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f180b3fa0 0x7f9f180b0130 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.337+0000 7f9f1e200640 1 --2- 172.21.15.82:0/3381198447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180b2e50 0x7f9f1815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.337+0000 7f9f1e200640 1 -- 172.21.15.82:0/3381198447 >> 172.21.15.82:0/3381198447 conn(0x7f9f180b17b0 msgr2=0x7f9f180ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.338+0000 7f9f1e200640 1 -- 172.21.15.82:0/3381198447 shutdown_connections 2024-03-16T18:03:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.338+0000 7f9f1e200640 1 -- 172.21.15.82:0/3381198447 wait complete. 2024-03-16T18:03:57.361 INFO:tasks.ceph:Scrubbing osd.4 2024-03-16T18:03:57.361 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-16T18:03:57.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.573+0000 7f505dc00640 1 Processor -- start 2024-03-16T18:03:57.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.573+0000 7f505dc00640 1 -- start start 2024-03-16T18:03:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.575+0000 7f505dc00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f505815cb30 0x7f505806c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.575+0000 7f505dc00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505815cfe0 con 0x7f5058159500 2024-03-16T18:03:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.575+0000 7f505dc00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50580cf5a0 con 0x7f50580ccb80 2024-03-16T18:03:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.575+0000 7f505dc00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50580b3770 con 0x7f505815cb30 2024-03-16T18:03:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.575+0000 7f5057e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5058159500 0x7f50580ced70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59010/0 (socket says 172.21.15.82:59010) 2024-03-16T18:03:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.575+0000 7f5057e00640 1 -- 172.21.15.82:0/2799769455 learned_addr learned my addr 172.21.15.82:0/2799769455 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:57.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.575+0000 7f504fe00640 1 --2- 172.21.15.82:0/2799769455 >> v2:172.21.15.82:3300/0 conn(0x7f505815cb30 0x7f505806c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.576+0000 7f5057400640 1 -- 172.21.15.82:0/2799769455 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941752367 0 0) 0x7f505815cfe0 con 0x7f5058159500 2024-03-16T18:03:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.576+0000 7f5057400640 1 -- 172.21.15.82:0/2799769455 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f503c003520 con 0x7f5058159500 2024-03-16T18:03:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.576+0000 7f504fe00640 1 -- 172.21.15.82:0/2799769455 >> v1:172.21.15.90:6789/0 conn(0x7f50580ccb80 legacy=0x7f50580ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.576+0000 7f504fe00640 1 -- 172.21.15.82:0/2799769455 >> v1:172.21.15.82:6789/0 conn(0x7f5058159500 legacy=0x7f50580ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.576+0000 7f504fe00640 1 -- 172.21.15.82:0/2799769455 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50580b3a50 con 0x7f505815cb30 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.577+0000 7f504fe00640 1 --2- 172.21.15.82:0/2799769455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f505815cb30 0x7f505806c8b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f50400097a0 tx=0x7f504002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0be78e1eeffc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.577+0000 7f5057400640 1 -- 172.21.15.82:0/2799769455 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5040030610 con 0x7f505815cb30 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.577+0000 7f5057400640 1 -- 172.21.15.82:0/2799769455 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5040030770 con 0x7f505815cb30 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.578+0000 7f5057400640 1 -- 172.21.15.82:0/2799769455 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5040030b00 con 0x7f505815cb30 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.578+0000 7f505dc00640 1 -- 172.21.15.82:0/2799769455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f505815cb30 msgr2=0x7f505806c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.578+0000 7f505dc00640 1 --2- 172.21.15.82:0/2799769455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f505815cb30 0x7f505806c8b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f50400097a0 tx=0x7f504002b960 comp rx=0 tx=0).stop 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.578+0000 7f505dc00640 1 -- 172.21.15.82:0/2799769455 shutdown_connections 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.578+0000 7f505dc00640 1 --2- 172.21.15.82:0/2799769455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f505815cb30 0x7f505806c8b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.578+0000 7f505dc00640 1 -- 172.21.15.82:0/2799769455 >> 172.21.15.82:0/2799769455 conn(0x7f505806b050 msgr2=0x7f505806b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.578+0000 7f505dc00640 1 -- 172.21.15.82:0/2799769455 shutdown_connections 2024-03-16T18:03:57.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.578+0000 7f505dc00640 1 -- 172.21.15.82:0/2799769455 wait complete. 2024-03-16T18:03:57.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.580+0000 7f505dc00640 1 Processor -- start 2024-03-16T18:03:57.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.580+0000 7f505dc00640 1 -- start start 2024-03-16T18:03:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.581+0000 7f505dc00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50580ccb80 0x7f505814a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.581+0000 7f505dc00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5058159500 0x7f5058157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.581+0000 7f505dc00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f505815cb30 0x7f50580ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.581+0000 7f505dc00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505814a720 con 0x7f505815cb30 2024-03-16T18:03:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.581+0000 7f505dc00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5058156800 con 0x7f5058159500 2024-03-16T18:03:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.581+0000 7f505dc00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5058156ae0 con 0x7f50580ccb80 2024-03-16T18:03:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.581+0000 7f505d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f505815cb30 0x7f50580ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.581+0000 7f505d200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f505815cb30 0x7f50580ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60206/0 (socket says 172.21.15.82:60206) 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.581+0000 7f505d200640 1 -- 172.21.15.82:0/2350777976 learned_addr learned my addr 172.21.15.82:0/2350777976 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f504fe00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5058159500 0x7f5058157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f5057e00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50580ccb80 0x7f505814a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f505d200640 1 -- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50580ccb80 msgr2=0x7f505814a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f505d200640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f50580ccb80 0x7f505814a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f505d200640 1 -- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5058159500 msgr2=0x7f5058157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f505d200640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5058159500 0x7f5058157ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f505d200640 1 -- 172.21.15.82:0/2350777976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5040009400 con 0x7f505815cb30 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f505d200640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f505815cb30 0x7f50580ad850 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f505000c7a0 tx=0x7f505000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f5054c00640 1 -- 172.21.15.82:0/2350777976 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050003b70 con 0x7f505815cb30 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f5054c00640 1 -- 172.21.15.82:0/2350777976 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5050003cf0 con 0x7f505815cb30 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.582+0000 7f5054c00640 1 -- 172.21.15.82:0/2350777976 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050003ea0 con 0x7f505815cb30 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.583+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5058159a60 con 0x7f505815cb30 2024-03-16T18:03:57.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.583+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50581570a0 con 0x7f505815cb30 2024-03-16T18:03:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.584+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f50581598e0 con 0x7f505815cb30 2024-03-16T18:03:57.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.585+0000 7f5054c00640 1 -- 172.21.15.82:0/2350777976 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f505000aa90 con 0x7f505815cb30 2024-03-16T18:03:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.585+0000 7f5054c00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f503406b3b0 0x7f503406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.586+0000 7f5054c00640 1 -- 172.21.15.82:0/2350777976 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5050089780 con 0x7f505815cb30 2024-03-16T18:03:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.586+0000 7f5057e00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f503406b3b0 0x7f503406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.586+0000 7f5054c00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] conn(0x7f5034071270 0x7f5034073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.586+0000 7f5054c00640 1 -- 172.21.15.82:0/2350777976 --> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5034073d40 con 0x7f5034071270 2024-03-16T18:03:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.586+0000 7f5054c00640 1 -- 172.21.15.82:0/2350777976 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f505008be40 con 0x7f505815cb30 2024-03-16T18:03:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.586+0000 7f5057e00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f503406b3b0 0x7f503406d870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f50480011e0 tx=0x7f5048009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:57.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.587+0000 7f504fe00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] conn(0x7f5034071270 0x7f5034073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.588+0000 7f504fe00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] conn(0x7f5034071270 0x7f5034073690 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:57.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.590+0000 7f5054c00640 1 -- 172.21.15.82:0/2350777976 <== osd.4 v2:172.21.15.90:6800/3078143209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5034073d40 con 0x7f5034071270 2024-03-16T18:03:57.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.604+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 --> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f50580b4010 con 0x7f5034071270 2024-03-16T18:03:57.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.605+0000 7f5054c00640 1 -- 172.21.15.82:0/2350777976 <== osd.4 v2:172.21.15.90:6800/3078143209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5040004040 con 0x7f5034071270 2024-03-16T18:03:57.609 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T18:03:57.609 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T18:03:57.609 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T18:03:57.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] conn(0x7f5034071270 msgr2=0x7f5034073690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505dc00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.90:6800/3078143209,v1:172.21.15.90:6801/3078143209] conn(0x7f5034071270 0x7f5034073690 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f503406b3b0 msgr2=0x7f503406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505dc00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f503406b3b0 0x7f503406d870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f50480011e0 tx=0x7f5048009040 comp rx=0 tx=0).stop 2024-03-16T18:03:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f505815cb30 msgr2=0x7f50580ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505dc00640 1 --2- 172.21.15.82:0/2350777976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f505815cb30 0x7f50580ad850 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f505000c7a0 tx=0x7f505000cc60 comp rx=0 tx=0).stop 2024-03-16T18:03:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505d200640 1 -- 172.21.15.82:0/2350777976 reap_dead start 2024-03-16T18:03:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 shutdown_connections 2024-03-16T18:03:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 >> 172.21.15.82:0/2350777976 conn(0x7f505806b050 msgr2=0x7f505815b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 shutdown_connections 2024-03-16T18:03:57.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.606+0000 7f505dc00640 1 -- 172.21.15.82:0/2350777976 wait complete. 2024-03-16T18:03:57.628 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-16T18:03:57.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.836+0000 7fd8bec00640 1 Processor -- start 2024-03-16T18:03:57.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.836+0000 7fd8bec00640 1 -- start start 2024-03-16T18:03:57.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.837+0000 7fd8bec00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd8b815ae00 0x7fd8b80b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.837+0000 7fd8bec00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8b8072590 0x7fd8b80b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.837+0000 7fd8bec00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b80cf610 con 0x7fd8b80729f0 2024-03-16T18:03:57.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.837+0000 7fd8bec00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b80cdaa0 con 0x7fd8b815ae00 2024-03-16T18:03:57.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.837+0000 7fd8bec00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b80cdd80 con 0x7fd8b8072590 2024-03-16T18:03:57.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.838+0000 7fd8b7e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd8b815ae00 0x7fd8b80b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.838+0000 7fd8b7e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fd8b815ae00 0x7fd8b80b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:55012/0 (socket says 172.21.15.82:55012) 2024-03-16T18:03:57.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.838+0000 7fd8b7400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8b8072590 0x7fd8b80b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.838+0000 7fd8b7e00640 1 -- 172.21.15.82:0/904304542 learned_addr learned my addr 172.21.15.82:0/904304542 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:57.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.838+0000 7fd8bcc00640 1 --1- >> v1:172.21.15.90:6789/0 conn(0x7fd8b80729f0 0x7fd8b806c870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.90:6789/0 says I am v1:172.21.15.82:54106/0 (socket says 172.21.15.82:54106) 2024-03-16T18:03:57.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.839+0000 7fd8b7e00640 1 -- 172.21.15.82:0/904304542 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8b8072590 msgr2=0x7fd8b80b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:57.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.839+0000 7fd8b7e00640 1 --2- 172.21.15.82:0/904304542 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8b8072590 0x7fd8b80b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.839+0000 7fd8b7e00640 1 -- 172.21.15.82:0/904304542 >> v1:172.21.15.90:6789/0 conn(0x7fd8b80729f0 legacy=0x7fd8b806c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.839+0000 7fd8b7e00640 1 -- 172.21.15.82:0/904304542 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b80cf2a0 con 0x7fd8b815ae00 2024-03-16T18:03:57.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.840+0000 7fd8b7e00640 1 --2- 172.21.15.82:0/904304542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b815ae00 0x7fd8b80b2e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009870 tx=0x7fd8a802db00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=400fc03013e7662f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:57.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.841+0000 7fd8b6a00640 1 -- 172.21.15.82:0/904304542 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a802f8c0 con 0x7fd8b815ae00 2024-03-16T18:03:57.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.841+0000 7fd8b6a00640 1 -- 172.21.15.82:0/904304542 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a802fa20 con 0x7fd8b815ae00 2024-03-16T18:03:57.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.841+0000 7fd8b6a00640 1 -- 172.21.15.82:0/904304542 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a802fdb0 con 0x7fd8b815ae00 2024-03-16T18:03:57.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.842+0000 7fd8bec00640 1 -- 172.21.15.82:0/904304542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b815ae00 msgr2=0x7fd8b80b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.842+0000 7fd8bec00640 1 --2- 172.21.15.82:0/904304542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b815ae00 0x7fd8b80b2e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009870 tx=0x7fd8a802db00 comp rx=0 tx=0).stop 2024-03-16T18:03:57.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.842+0000 7fd8bec00640 1 -- 172.21.15.82:0/904304542 shutdown_connections 2024-03-16T18:03:57.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.842+0000 7fd8bec00640 1 --2- 172.21.15.82:0/904304542 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8b8072590 0x7fd8b80b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.842+0000 7fd8bec00640 1 --2- 172.21.15.82:0/904304542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b815ae00 0x7fd8b80b2e50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.842+0000 7fd8bec00640 1 -- 172.21.15.82:0/904304542 >> 172.21.15.82:0/904304542 conn(0x7fd8b80b17b0 msgr2=0x7fd8b80b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:57.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.842+0000 7fd8bec00640 1 -- 172.21.15.82:0/904304542 shutdown_connections 2024-03-16T18:03:57.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.842+0000 7fd8bec00640 1 -- 172.21.15.82:0/904304542 wait complete. 2024-03-16T18:03:57.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.843+0000 7fd8bec00640 1 Processor -- start 2024-03-16T18:03:57.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.844+0000 7fd8bec00640 1 -- start start 2024-03-16T18:03:57.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.844+0000 7fd8bec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8b8072590 0x7fd8b815c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8bec00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b80729f0 0x7fd8b815db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8bec00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8b815ae00 0x7fd8b815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8bec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b815c010 con 0x7fd8b8072590 2024-03-16T18:03:57.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8bec00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b815b3d0 con 0x7fd8b80729f0 2024-03-16T18:03:57.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8bec00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b814e890 con 0x7fd8b815ae00 2024-03-16T18:03:57.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8b7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8b8072590 0x7fd8b815c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8b7e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8b8072590 0x7fd8b815c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60214/0 (socket says 172.21.15.82:60214) 2024-03-16T18:03:57.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8b7e00640 1 -- 172.21.15.82:0/910562131 learned_addr learned my addr 172.21.15.82:0/910562131 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:57.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8b7400640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b80729f0 0x7fd8b815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8bcc00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8b815ae00 0x7fd8b815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8b7e00640 1 -- 172.21.15.82:0/910562131 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8b815ae00 msgr2=0x7fd8b815e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8b7e00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8b815ae00 0x7fd8b815e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8b7e00640 1 -- 172.21.15.82:0/910562131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b80729f0 msgr2=0x7fd8b815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:57.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8b7e00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b80729f0 0x7fd8b815db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:57.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.845+0000 7fd8b7e00640 1 -- 172.21.15.82:0/910562131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80094d0 con 0x7fd8b8072590 2024-03-16T18:03:57.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.846+0000 7fd8b7e00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8b8072590 0x7fd8b815c430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd8a802db40 tx=0x7fd8a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:57.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.846+0000 7fd8afe00640 1 -- 172.21.15.82:0/910562131 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a80408f0 con 0x7fd8b8072590 2024-03-16T18:03:57.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.846+0000 7fd8afe00640 1 -- 172.21.15.82:0/910562131 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8040a90 con 0x7fd8b8072590 2024-03-16T18:03:57.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.846+0000 7fd8afe00640 1 -- 172.21.15.82:0/910562131 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8040e40 con 0x7fd8b8072590 2024-03-16T18:03:57.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.846+0000 7fd8bec00640 1 -- 172.21.15.82:0/910562131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b815b670 con 0x7fd8b8072590 2024-03-16T18:03:57.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.846+0000 7fd8bec00640 1 -- 172.21.15.82:0/910562131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b815bc30 con 0x7fd8b8072590 2024-03-16T18:03:57.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.847+0000 7fd8bec00640 1 -- 172.21.15.82:0/910562131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b815bdc0 con 0x7fd8b8072590 2024-03-16T18:03:57.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.848+0000 7fd8afe00640 1 -- 172.21.15.82:0/910562131 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fd8a8037420 con 0x7fd8b8072590 2024-03-16T18:03:57.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.849+0000 7fd8afe00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd89806b3b0 0x7fd89806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:57.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.849+0000 7fd8afe00640 1 -- 172.21.15.82:0/910562131 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd8a8033070 con 0x7fd8b8072590 2024-03-16T18:03:57.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.849+0000 7fd8b7400640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd89806b3b0 0x7fd89806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:57.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.850+0000 7fd8b7400640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd89806b3b0 0x7fd89806d870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8155390 tx=0x7fd8a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:57.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:57.853+0000 7fd8afe00640 1 -- 172.21.15.82:0/910562131 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd8a8078bf0 con 0x7fd8b8072590 2024-03-16T18:03:58.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.045+0000 7fd8bec00640 1 -- 172.21.15.82:0/910562131 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8b815c280 con 0x7fd89806b3b0 2024-03-16T18:03:58.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.046+0000 7fd8afe00640 1 -- 172.21.15.82:0/910562131 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd8b815c280 con 0x7fd89806b3b0 2024-03-16T18:03:58.050 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 4 to deep-scrub 2024-03-16T18:03:58.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.048+0000 7fd8bec00640 1 -- 172.21.15.82:0/910562131 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd89806b3b0 msgr2=0x7fd89806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.048+0000 7fd8bec00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd89806b3b0 0x7fd89806d870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8155390 tx=0x7fd8a4008040 comp rx=0 tx=0).stop 2024-03-16T18:03:58.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.048+0000 7fd8bec00640 1 -- 172.21.15.82:0/910562131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8b8072590 msgr2=0x7fd8b815c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.048+0000 7fd8bec00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8b8072590 0x7fd8b815c430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd8a802db40 tx=0x7fd8a80046c0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.048+0000 7fd8bec00640 1 -- 172.21.15.82:0/910562131 shutdown_connections 2024-03-16T18:03:58.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.048+0000 7fd8bec00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fd89806b3b0 0x7fd89806d870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.048+0000 7fd8bec00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fd8b815ae00 0x7fd8b815e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.048+0000 7fd8bec00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd8b80729f0 0x7fd8b815db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.048+0000 7fd8bec00640 1 --2- 172.21.15.82:0/910562131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8b8072590 0x7fd8b815c430 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.048+0000 7fd8bec00640 1 -- 172.21.15.82:0/910562131 >> 172.21.15.82:0/910562131 conn(0x7fd8b80b17b0 msgr2=0x7fd8b815f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:58.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.049+0000 7fd8bec00640 1 -- 172.21.15.82:0/910562131 shutdown_connections 2024-03-16T18:03:58.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.049+0000 7fd8bec00640 1 -- 172.21.15.82:0/910562131 wait complete. 2024-03-16T18:03:58.071 INFO:tasks.ceph:Scrubbing osd.5 2024-03-16T18:03:58.071 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-16T18:03:58.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.269+0000 7f5e81800640 1 Processor -- start 2024-03-16T18:03:58.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.269+0000 7f5e81800640 1 -- start start 2024-03-16T18:03:58.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.270+0000 7f5e81800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5e7c06dc70 0x7f5e7c06e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.270+0000 7f5e81800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5e7c0af050 0x7f5e7c0b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.270+0000 7f5e81800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e7c06e590 con 0x7f5e7c15e150 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.270+0000 7f5e81800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c0cf720 con 0x7f5e7c0af050 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.270+0000 7f5e81800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c0cc530 con 0x7f5e7c06dc70 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.270+0000 7f5e7b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5e7c0af050 0x7f5e7c0b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.270+0000 7f5e7b400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5e7c0af050 0x7f5e7c0b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60230/0 (socket says 172.21.15.82:60230) 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.270+0000 7f5e7b400640 1 -- 172.21.15.82:0/2993693049 learned_addr learned my addr 172.21.15.82:0/2993693049 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.271+0000 7f5e7aa00640 1 --2- 172.21.15.82:0/2993693049 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5e7c06dc70 0x7f5e7c06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.271+0000 7f5e7b400640 1 -- 172.21.15.82:0/2993693049 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5e7c06dc70 msgr2=0x7f5e7c06e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.271+0000 7f5e7b400640 1 --2- 172.21.15.82:0/2993693049 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5e7c06dc70 0x7f5e7c06e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.271+0000 7f5e7b400640 1 -- 172.21.15.82:0/2993693049 >> v1:172.21.15.90:6789/0 conn(0x7f5e7c15e150 legacy=0x7f5e7c1624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.271+0000 7f5e7b400640 1 -- 172.21.15.82:0/2993693049 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c0cc810 con 0x7f5e7c0af050 2024-03-16T18:03:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.271+0000 7f5e7aa00640 1 --2- 172.21.15.82:0/2993693049 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5e7c06dc70 0x7f5e7c06e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.272+0000 7f5e7b400640 1 --2- 172.21.15.82:0/2993693049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c0af050 0x7f5e7c0b2a40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009e50 tx=0x7f5e7402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401990d49c81e346 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.272+0000 7f5e79600640 1 -- 172.21.15.82:0/2993693049 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7402c9e0 con 0x7f5e7c0af050 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.272+0000 7f5e79600640 1 -- 172.21.15.82:0/2993693049 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7402cb40 con 0x7f5e7c0af050 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.272+0000 7f5e79600640 1 -- 172.21.15.82:0/2993693049 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e740343f0 con 0x7f5e7c0af050 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.273+0000 7f5e81800640 1 -- 172.21.15.82:0/2993693049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c0af050 msgr2=0x7f5e7c0b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.273+0000 7f5e81800640 1 --2- 172.21.15.82:0/2993693049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c0af050 0x7f5e7c0b2a40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009e50 tx=0x7f5e7402be40 comp rx=0 tx=0).stop 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.273+0000 7f5e81800640 1 -- 172.21.15.82:0/2993693049 shutdown_connections 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.273+0000 7f5e81800640 1 --2- 172.21.15.82:0/2993693049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c0af050 0x7f5e7c0b2a40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.273+0000 7f5e81800640 1 --2- 172.21.15.82:0/2993693049 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5e7c06dc70 0x7f5e7c06e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.273+0000 7f5e81800640 1 -- 172.21.15.82:0/2993693049 >> 172.21.15.82:0/2993693049 conn(0x7f5e7c0b1740 msgr2=0x7f5e7c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.273+0000 7f5e81800640 1 -- 172.21.15.82:0/2993693049 shutdown_connections 2024-03-16T18:03:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.273+0000 7f5e81800640 1 -- 172.21.15.82:0/2993693049 wait complete. 2024-03-16T18:03:58.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.275+0000 7f5e81800640 1 Processor -- start 2024-03-16T18:03:58.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.275+0000 7f5e81800640 1 -- start start 2024-03-16T18:03:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e81800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5e7c06dc70 0x7f5e7c0c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e81800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e7c0af050 0x7f5e7c15e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e81800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c15e150 0x7f5e7c14af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e81800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c149be0 con 0x7f5e7c15e150 2024-03-16T18:03:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e81800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c148fd0 con 0x7f5e7c0af050 2024-03-16T18:03:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e81800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c1492b0 con 0x7f5e7c06dc70 2024-03-16T18:03:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c15e150 0x7f5e7c14af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7b400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c15e150 0x7f5e7c14af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60242/0 (socket says 172.21.15.82:60242) 2024-03-16T18:03:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7b400640 1 -- 172.21.15.82:0/3239686461 learned_addr learned my addr 172.21.15.82:0/3239686461 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7a000640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e7c0af050 0x7f5e7c15e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7aa00640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5e7c06dc70 0x7f5e7c0c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7b400640 1 -- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5e7c06dc70 msgr2=0x7f5e7c0c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7b400640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f5e7c06dc70 0x7f5e7c0c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7b400640 1 -- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e7c0af050 msgr2=0x7f5e7c15e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7b400640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e7c0af050 0x7f5e7c15e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7b400640 1 -- 172.21.15.82:0/3239686461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74009b30 con 0x7f5e7c15e150 2024-03-16T18:03:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.276+0000 7f5e7a000640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e7c0af050 0x7f5e7c15e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.277+0000 7f5e7b400640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c15e150 0x7f5e7c14af60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5e74036040 tx=0x7f5e74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.277+0000 7f5e72a00640 1 -- 172.21.15.82:0/3239686461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7403e450 con 0x7f5e7c15e150 2024-03-16T18:03:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.277+0000 7f5e72a00640 1 -- 172.21.15.82:0/3239686461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7403e5d0 con 0x7f5e7c15e150 2024-03-16T18:03:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.277+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7c149590 con 0x7f5e7c15e150 2024-03-16T18:03:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.277+0000 7f5e72a00640 1 -- 172.21.15.82:0/3239686461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7403e980 con 0x7f5e7c15e150 2024-03-16T18:03:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.277+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c0c4ef0 con 0x7f5e7c15e150 2024-03-16T18:03:58.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.278+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e7c0cd0b0 con 0x7f5e7c15e150 2024-03-16T18:03:58.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.279+0000 7f5e72a00640 1 -- 172.21.15.82:0/3239686461 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f5e7403eae0 con 0x7f5e7c15e150 2024-03-16T18:03:58.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.280+0000 7f5e72a00640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5e5c06b3b0 0x7f5e5c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.280+0000 7f5e72a00640 1 -- 172.21.15.82:0/3239686461 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5e740ac840 con 0x7f5e7c15e150 2024-03-16T18:03:58.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.280+0000 7f5e7aa00640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5e5c06b3b0 0x7f5e5c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.280+0000 7f5e72a00640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] conn(0x7f5e5c071270 0x7f5e5c073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.280+0000 7f5e72a00640 1 -- 172.21.15.82:0/3239686461 --> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e5c073d40 con 0x7f5e5c071270 2024-03-16T18:03:58.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.280+0000 7f5e72a00640 1 -- 172.21.15.82:0/3239686461 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e740aee20 con 0x7f5e7c15e150 2024-03-16T18:03:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.281+0000 7f5e7aa00640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5e5c06b3b0 0x7f5e5c06d870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c008850 tx=0x7f5e6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.281+0000 7f5e7a000640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] conn(0x7f5e5c071270 0x7f5e5c073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.282+0000 7f5e7a000640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] conn(0x7f5e5c071270 0x7f5e5c073690 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.284+0000 7f5e72a00640 1 -- 172.21.15.82:0/3239686461 <== osd.5 v2:172.21.15.90:6808/2407665501 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5e5c073d40 con 0x7f5e5c071270 2024-03-16T18:03:58.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.298+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 --> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5e7c0ccdd0 con 0x7f5e5c071270 2024-03-16T18:03:58.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.299+0000 7f5e72a00640 1 -- 172.21.15.82:0/3239686461 <== osd.5 v2:172.21.15.90:6808/2407665501 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5e6800a960 con 0x7f5e5c071270 2024-03-16T18:03:58.303 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T18:03:58.303 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T18:03:58.303 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T18:03:58.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.300+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] conn(0x7f5e5c071270 msgr2=0x7f5e5c073690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.300+0000 7f5e81800640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.90:6808/2407665501,v1:172.21.15.90:6809/2407665501] conn(0x7f5e5c071270 0x7f5e5c073690 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.300+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5e5c06b3b0 msgr2=0x7f5e5c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.300+0000 7f5e81800640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f5e5c06b3b0 0x7f5e5c06d870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c008850 tx=0x7f5e6c008040 comp rx=0 tx=0).stop 2024-03-16T18:03:58.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.300+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c15e150 msgr2=0x7f5e7c14af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.300+0000 7f5e81800640 1 --2- 172.21.15.82:0/3239686461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c15e150 0x7f5e7c14af60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5e74036040 tx=0x7f5e74004a40 comp rx=0 tx=0).stop 2024-03-16T18:03:58.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.300+0000 7f5e7b400640 1 -- 172.21.15.82:0/3239686461 reap_dead start 2024-03-16T18:03:58.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.300+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 shutdown_connections 2024-03-16T18:03:58.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.300+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 >> 172.21.15.82:0/3239686461 conn(0x7f5e7c0b1740 msgr2=0x7f5e7c0cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:58.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.301+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 shutdown_connections 2024-03-16T18:03:58.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.301+0000 7f5e81800640 1 -- 172.21.15.82:0/3239686461 wait complete. 2024-03-16T18:03:58.320 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-16T18:03:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.510+0000 7fcab7200640 1 Processor -- start 2024-03-16T18:03:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.511+0000 7fcab7200640 1 -- start start 2024-03-16T18:03:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.512+0000 7fcab7200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab0072240 0x7fcab0072690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.512+0000 7fcab7200640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab0072bd0 con 0x7fcab00aee10 2024-03-16T18:03:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.512+0000 7fcab7200640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab00b3130 con 0x7fcab0144330 2024-03-16T18:03:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.512+0000 7fcab7200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00b39a0 con 0x7fcab0072240 2024-03-16T18:03:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.512+0000 7fcab5400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcab00aee10 0x7fcab00ad850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59020/0 (socket says 172.21.15.82:59020) 2024-03-16T18:03:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.512+0000 7fcab5400640 1 -- 172.21.15.82:0/2399147982 learned_addr learned my addr 172.21.15.82:0/2399147982 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.513+0000 7fcab6800640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab0072240 0x7fcab0072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.513+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349956135 0 0) 0x7fcab0072bd0 con 0x7fcab00aee10 2024-03-16T18:03:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.513+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca98003520 con 0x7fcab00aee10 2024-03-16T18:03:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.513+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 204256674 0 0) 0x7fca98003520 con 0x7fcab00aee10 2024-03-16T18:03:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.513+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab0072240 msgr2=0x7fcab0072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.513+0000 7fcab4a00640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab0072240 0x7fcab0072690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.513+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 >> v1:172.21.15.90:6789/0 conn(0x7fcab0144330 legacy=0x7fcab00b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.513+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00b33e0 con 0x7fcab00aee10 2024-03-16T18:03:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.513+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fcaa0001690 con 0x7fcab00aee10 2024-03-16T18:03:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.513+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 >> v1:172.21.15.82:6789/0 conn(0x7fcab00aee10 legacy=0x7fcab00ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab4a00640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab0072240 0x7fca980039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab4a00640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98003f20 0x7fca980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab4a00640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca98004d00 0x7fca98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00b39a0 con 0x7fca98004d00 2024-03-16T18:03:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9800ba60 con 0x7fca98003f20 2024-03-16T18:03:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9800bc20 con 0x7fcab0072240 2024-03-16T18:03:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab6800640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca98004d00 0x7fca98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab5400640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98003f20 0x7fca980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab5e00640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab0072240 0x7fca980039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab6800640 1 -- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab0072240 msgr2=0x7fca980039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab6800640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab0072240 0x7fca980039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab6800640 1 -- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98003f20 msgr2=0x7fca980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab6800640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98003f20 0x7fca980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab6800640 1 -- 172.21.15.82:0/2399147982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca98003520 con 0x7fca98004d00 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab5400640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98003f20 0x7fca980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.514+0000 7fcab5e00640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab0072240 0x7fca980039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.515+0000 7fcab6800640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca98004d00 0x7fca98005180 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009fc0 tx=0x7fca980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70100eee9c86196 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.516+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac02cc20 con 0x7fca98004d00 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.516+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac00f040 con 0x7fca98004d00 2024-03-16T18:03:58.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.516+0000 7fcab4a00640 1 -- 172.21.15.82:0/2399147982 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac034440 con 0x7fca98004d00 2024-03-16T18:03:58.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.516+0000 7fcab7200640 1 -- 172.21.15.82:0/2399147982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca98004d00 msgr2=0x7fca98005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.516+0000 7fcab7200640 1 --2- 172.21.15.82:0/2399147982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca98004d00 0x7fca98005180 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009fc0 tx=0x7fca980057f0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.516+0000 7fcab6800640 1 -- 172.21.15.82:0/2399147982 reap_dead start 2024-03-16T18:03:58.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.516+0000 7fcab7200640 1 -- 172.21.15.82:0/2399147982 shutdown_connections 2024-03-16T18:03:58.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.516+0000 7fcab7200640 1 -- 172.21.15.82:0/2399147982 >> 172.21.15.82:0/2399147982 conn(0x7fcab006ae70 msgr2=0x7fcab006b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:58.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.516+0000 7fcab7200640 1 -- 172.21.15.82:0/2399147982 shutdown_connections 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.516+0000 7fcab7200640 1 -- 172.21.15.82:0/2399147982 wait complete. 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.518+0000 7fcab7200640 1 Processor -- start 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.518+0000 7fcab7200640 1 -- start start 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.519+0000 7fcab7200640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98004d00 0x7fcab01f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.519+0000 7fcab7200640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab015c140 0x7fcab0142ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.519+0000 7fcab7200640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab015c5c0 0x7fcab0143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.519+0000 7fcab7200640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00cdae0 con 0x7fcab015c140 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.519+0000 7fcab7200640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00cdd30 con 0x7fca98004d00 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.519+0000 7fcab7200640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab00ce010 con 0x7fcab015c5c0 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.519+0000 7fcab5400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab015c140 0x7fcab0142ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.519+0000 7fcab5400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab015c140 0x7fcab0142ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60268/0 (socket says 172.21.15.82:60268) 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.519+0000 7fcab5400640 1 -- 172.21.15.82:0/856935841 learned_addr learned my addr 172.21.15.82:0/856935841 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:58.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcab5e00640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98004d00 0x7fcab01f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcab6800640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab015c5c0 0x7fcab0143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcab5400640 1 -- 172.21.15.82:0/856935841 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab015c5c0 msgr2=0x7fcab0143410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcab5400640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab015c5c0 0x7fcab0143410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcab5400640 1 -- 172.21.15.82:0/856935841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98004d00 msgr2=0x7fcab01f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcab5400640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98004d00 0x7fcab01f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcab5400640 1 -- 172.21.15.82:0/856935841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00ce2f0 con 0x7fcab015c140 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcab5e00640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98004d00 0x7fcab01f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcab5400640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab015c140 0x7fcab0142ed0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000d8e0 tx=0x7fcaa000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcaaa000640 1 -- 172.21.15.82:0/856935841 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00075a0 con 0x7fcab015c140 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcaaa000640 1 -- 172.21.15.82:0/856935841 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa00047e0 con 0x7fcab015c140 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcaaa000640 1 -- 172.21.15.82:0/856935841 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0004b70 con 0x7fcab015c140 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.520+0000 7fcab7200640 1 -- 172.21.15.82:0/856935841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab00ce5d0 con 0x7fcab015c140 2024-03-16T18:03:58.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.521+0000 7fcab7200640 1 -- 172.21.15.82:0/856935841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab00c5460 con 0x7fcab015c140 2024-03-16T18:03:58.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.522+0000 7fcab7200640 1 -- 172.21.15.82:0/856935841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab00ce760 con 0x7fcab015c140 2024-03-16T18:03:58.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.522+0000 7fcaaa000640 1 -- 172.21.15.82:0/856935841 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fcaa0004cd0 con 0x7fcab015c140 2024-03-16T18:03:58.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.524+0000 7fcaaa000640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fca9406b3b0 0x7fca9406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.524+0000 7fcaaa000640 1 -- 172.21.15.82:0/856935841 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcaa00894a0 con 0x7fcab015c140 2024-03-16T18:03:58.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.524+0000 7fcab5e00640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fca9406b3b0 0x7fca9406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.524+0000 7fcab5e00640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fca9406b3b0 0x7fca9406d870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40016f0 tx=0x7fcaa4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.528+0000 7fcaaa000640 1 -- 172.21.15.82:0/856935841 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcaa00543a0 con 0x7fcab015c140 2024-03-16T18:03:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.721+0000 7fcab7200640 1 -- 172.21.15.82:0/856935841 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fcab0071d20 con 0x7fca9406b3b0 2024-03-16T18:03:58.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.722+0000 7fcaaa000640 1 -- 172.21.15.82:0/856935841 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcaa400d040 con 0x7fca9406b3b0 2024-03-16T18:03:58.725 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 5 to deep-scrub 2024-03-16T18:03:58.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 -- 172.21.15.82:0/856935841 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fca9406b3b0 msgr2=0x7fca9406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fca9406b3b0 0x7fca9406d870 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcaa40016f0 tx=0x7fcaa4009040 comp rx=0 tx=0).stop 2024-03-16T18:03:58.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 -- 172.21.15.82:0/856935841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab015c140 msgr2=0x7fcab0142ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab015c140 0x7fcab0142ed0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000d8e0 tx=0x7fcaa000dda0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 -- 172.21.15.82:0/856935841 shutdown_connections 2024-03-16T18:03:58.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fca9406b3b0 0x7fca9406d870 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fcab015c5c0 0x7fcab0143410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcab015c140 0x7fcab0142ed0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 --2- 172.21.15.82:0/856935841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fca98004d00 0x7fcab01f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 -- 172.21.15.82:0/856935841 >> 172.21.15.82:0/856935841 conn(0x7fcab006ae70 msgr2=0x7fcab0161810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:58.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 -- 172.21.15.82:0/856935841 shutdown_connections 2024-03-16T18:03:58.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.724+0000 7fcab7200640 1 -- 172.21.15.82:0/856935841 wait complete. 2024-03-16T18:03:58.746 INFO:tasks.ceph:Scrubbing osd.6 2024-03-16T18:03:58.746 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-16T18:03:58.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.952+0000 7f8bac000640 1 Processor -- start 2024-03-16T18:03:58.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.953+0000 7f8bac000640 1 -- start start 2024-03-16T18:03:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.954+0000 7f8bac000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40d4aa0 0x7f8ba40d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.954+0000 7f8bac000640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba40b3e40 con 0x7f8ba40d2630 2024-03-16T18:03:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.954+0000 7f8bac000640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba40b3510 con 0x7f8ba415e160 2024-03-16T18:03:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.954+0000 7f8bac000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba40cfd10 con 0x7f8ba40d4aa0 2024-03-16T18:03:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.954+0000 7f8baa000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8ba40d2630 0x7f8ba40b42d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59024/0 (socket says 172.21.15.82:59024) 2024-03-16T18:03:58.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.954+0000 7f8baa000640 1 -- 172.21.15.82:0/927914955 learned_addr learned my addr 172.21.15.82:0/927914955 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba8c00640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40d4aa0 0x7f8ba40d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209055320 0 0) 0x7f8ba40b3e40 con 0x7f8ba40d2630 2024-03-16T18:03:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b88003520 con 0x7f8ba40d2630 2024-03-16T18:03:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 199983976 0 0) 0x7f8b88003520 con 0x7f8ba40d2630 2024-03-16T18:03:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40d4aa0 msgr2=0x7f8ba40d20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40d4aa0 0x7f8ba40d20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 >> v1:172.21.15.90:6789/0 conn(0x7f8ba415e160 legacy=0x7f8ba40cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba40b37f0 con 0x7f8ba40d2630 2024-03-16T18:03:58.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7f8b9c0019f0 con 0x7f8ba40d2630 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 >> v1:172.21.15.82:6789/0 conn(0x7f8ba40d2630 legacy=0x7f8ba40b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40d4aa0 0x7f8b880039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b88004020 0x7f8b880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 0x7f8b88005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba40cfd10 con 0x7f8b88004d00 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8800bbc0 con 0x7f8b88004020 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.955+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8800bd80 con 0x7f8ba40d4aa0 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.956+0000 7f8baa000640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 0x7f8b88005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.956+0000 7f8ba9600640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40d4aa0 0x7f8b880039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.956+0000 7f8ba8c00640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b88004020 0x7f8b880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.956+0000 7f8baa000640 1 -- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40d4aa0 msgr2=0x7f8b880039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.956+0000 7f8baa000640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40d4aa0 0x7f8b880039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.956+0000 7f8baa000640 1 -- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b88004020 msgr2=0x7f8b880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:58.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.956+0000 7f8baa000640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b88004020 0x7f8b880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.956+0000 7f8baa000640 1 -- 172.21.15.82:0/927914955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b88003520 con 0x7f8b88004d00 2024-03-16T18:03:58.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.957+0000 7f8baa000640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 0x7f8b88005180 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00a760 tx=0x7f8b880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e215462c3d37b5d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.957+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c034630 con 0x7f8b88004d00 2024-03-16T18:03:58.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.957+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c02f070 con 0x7f8b88004d00 2024-03-16T18:03:58.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.957+0000 7f8ba3e00640 1 -- 172.21.15.82:0/927914955 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c034a50 con 0x7f8b88004d00 2024-03-16T18:03:58.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.958+0000 7f8bac000640 1 -- 172.21.15.82:0/927914955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 msgr2=0x7f8b88005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.958+0000 7f8bac000640 1 --2- 172.21.15.82:0/927914955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 0x7f8b88005180 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00a760 tx=0x7f8b880057f0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.958+0000 7f8baa000640 1 -- 172.21.15.82:0/927914955 reap_dead start 2024-03-16T18:03:58.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.958+0000 7f8bac000640 1 -- 172.21.15.82:0/927914955 shutdown_connections 2024-03-16T18:03:58.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.958+0000 7f8bac000640 1 -- 172.21.15.82:0/927914955 >> 172.21.15.82:0/927914955 conn(0x7f8ba40d3dc0 msgr2=0x7f8ba40d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:58.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.958+0000 7f8bac000640 1 -- 172.21.15.82:0/927914955 shutdown_connections 2024-03-16T18:03:58.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.958+0000 7f8bac000640 1 -- 172.21.15.82:0/927914955 wait complete. 2024-03-16T18:03:58.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.960+0000 7f8bac000640 1 Processor -- start 2024-03-16T18:03:58.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.960+0000 7f8bac000640 1 -- start start 2024-03-16T18:03:58.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.961+0000 7f8bac000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 0x7f8ba40b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.961+0000 7f8bac000640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40cc970 0x7f8ba40b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.961+0000 7f8bac000640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba4157fe0 0x7f8ba40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.961+0000 7f8bac000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba40d2ae0 con 0x7f8b88004d00 2024-03-16T18:03:58.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.961+0000 7f8bac000640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba40b7c60 con 0x7f8ba4157fe0 2024-03-16T18:03:58.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.961+0000 7f8bac000640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba40b7050 con 0x7f8ba40cc970 2024-03-16T18:03:58.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.961+0000 7f8ba9600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 0x7f8ba40b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.961+0000 7f8ba9600640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 0x7f8ba40b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60282/0 (socket says 172.21.15.82:60282) 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.961+0000 7f8ba9600640 1 -- 172.21.15.82:0/2691070826 learned_addr learned my addr 172.21.15.82:0/2691070826 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8baa000640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba4157fe0 0x7f8ba40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8ba8c00640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40cc970 0x7f8ba40b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8ba9600640 1 -- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40cc970 msgr2=0x7f8ba40b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8ba9600640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f8ba40cc970 0x7f8ba40b80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8ba9600640 1 -- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba4157fe0 msgr2=0x7f8ba40b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8ba9600640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8ba4157fe0 0x7f8ba40b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8ba9600640 1 -- 172.21.15.82:0/2691070826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba40b7330 con 0x7f8b88004d00 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8ba9600640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 0x7f8ba40b69e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400e4d0 tx=0x7f8b9400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8ba1600640 1 -- 172.21.15.82:0/2691070826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b940053a0 con 0x7f8b88004d00 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8ba1600640 1 -- 172.21.15.82:0/2691070826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b940035f0 con 0x7f8b88004d00 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba40b7610 con 0x7f8b88004d00 2024-03-16T18:03:58.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8ba1600640 1 -- 172.21.15.82:0/2691070826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94003980 con 0x7f8b88004d00 2024-03-16T18:03:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.962+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba40b8630 con 0x7f8b88004d00 2024-03-16T18:03:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.963+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ba40b2c20 con 0x7f8b88004d00 2024-03-16T18:03:58.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.964+0000 7f8ba1600640 1 -- 172.21.15.82:0/2691070826 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f8b94003ae0 con 0x7f8b88004d00 2024-03-16T18:03:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.965+0000 7f8ba1600640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b8406b3b0 0x7f8b8406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.965+0000 7f8ba1600640 1 -- 172.21.15.82:0/2691070826 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8b94089670 con 0x7f8b88004d00 2024-03-16T18:03:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.965+0000 7f8ba8c00640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b8406b3b0 0x7f8b8406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.966+0000 7f8ba1600640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] conn(0x7f8b84071270 0x7f8b84073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.966+0000 7f8ba1600640 1 -- 172.21.15.82:0/2691070826 --> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b84073d40 con 0x7f8b84071270 2024-03-16T18:03:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.966+0000 7f8ba1600640 1 -- 172.21.15.82:0/2691070826 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b9408bc50 con 0x7f8b88004d00 2024-03-16T18:03:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.966+0000 7f8ba8c00640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b8406b3b0 0x7f8b8406d870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ba41f73b0 tx=0x7f8b90008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.966+0000 7f8baa000640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] conn(0x7f8b84071270 0x7f8b84073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:58.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.967+0000 7f8baa000640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] conn(0x7f8b84071270 0x7f8b84073690 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:58.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.970+0000 7f8ba1600640 1 -- 172.21.15.82:0/2691070826 <== osd.6 v2:172.21.15.90:6816/1201371759 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8b84073d40 con 0x7f8b84071270 2024-03-16T18:03:58.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.984+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 --> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8ba40cfaf0 con 0x7f8b84071270 2024-03-16T18:03:58.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.985+0000 7f8ba1600640 1 -- 172.21.15.82:0/2691070826 <== osd.6 v2:172.21.15.90:6816/1201371759 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8b9c001250 con 0x7f8b84071270 2024-03-16T18:03:58.989 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T18:03:58.989 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T18:03:58.989 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] conn(0x7f8b84071270 msgr2=0x7f8b84073690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8bac000640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.90:6816/1201371759,v1:172.21.15.90:6818/1201371759] conn(0x7f8b84071270 0x7f8b84073690 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b8406b3b0 msgr2=0x7f8b8406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8bac000640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f8b8406b3b0 0x7f8b8406d870 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8ba41f73b0 tx=0x7f8b90008040 comp rx=0 tx=0).stop 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 msgr2=0x7f8ba40b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8bac000640 1 --2- 172.21.15.82:0/2691070826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b88004d00 0x7f8ba40b69e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400e4d0 tx=0x7f8b9400e990 comp rx=0 tx=0).stop 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8baa000640 1 -- 172.21.15.82:0/2691070826 reap_dead start 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 shutdown_connections 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 >> 172.21.15.82:0/2691070826 conn(0x7f8ba40d3dc0 msgr2=0x7f8ba40d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 shutdown_connections 2024-03-16T18:03:58.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:58.986+0000 7f8bac000640 1 -- 172.21.15.82:0/2691070826 wait complete. 2024-03-16T18:03:59.006 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-16T18:03:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.203+0000 7f94b3e00640 1 Processor -- start 2024-03-16T18:03:59.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.204+0000 7f94b3e00640 1 -- start start 2024-03-16T18:03:59.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.205+0000 7f94b3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f94ac15c120 0x7f94ac0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.205+0000 7f94b3e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0d08b0 0x7f94ac0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.205+0000 7f94b3e00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ac0cc7f0 con 0x7f94ac0cf150 2024-03-16T18:03:59.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.205+0000 7f94b3e00640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ac0cfd60 con 0x7f94ac15c120 2024-03-16T18:03:59.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.205+0000 7f94b3e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0cf5e0 con 0x7f94ac0d08b0 2024-03-16T18:03:59.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.206+0000 7f94b1400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f94ac15c120 0x7f94ac0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.206+0000 7f94b1e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0d08b0 0x7f94ac0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.206+0000 7f94b1400640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f94ac15c120 0x7f94ac0cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:55074/0 (socket says 172.21.15.82:55074) 2024-03-16T18:03:59.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.206+0000 7f94b1400640 1 -- 172.21.15.82:0/2366099099 learned_addr learned my addr 172.21.15.82:0/2366099099 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:59.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.207+0000 7f94b1400640 1 -- 172.21.15.82:0/2366099099 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0d08b0 msgr2=0x7f94ac0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:59.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.207+0000 7f94b1400640 1 --2- 172.21.15.82:0/2366099099 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0d08b0 0x7f94ac0cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.207+0000 7f94b1400640 1 -- 172.21.15.82:0/2366099099 >> v1:172.21.15.90:6789/0 conn(0x7f94ac0cf150 legacy=0x7f94ac0d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.207+0000 7f94b1400640 1 -- 172.21.15.82:0/2366099099 --> v2:172.21.15.90:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac0cff70 con 0x7f94ac15c120 2024-03-16T18:03:59.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.207+0000 7f94b1e00640 1 --2- 172.21.15.82:0/2366099099 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0d08b0 0x7f94ac0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T18:03:59.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.208+0000 7f94b1400640 1 --2- 172.21.15.82:0/2366099099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94ac15c120 0x7f94ac0cd7e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94a00098a0 tx=0x7f94a002ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30ed92c9b6ca403b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:59.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.209+0000 7f94abe00640 1 -- 172.21.15.82:0/2366099099 <== mon.1 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a002f830 con 0x7f94ac15c120 2024-03-16T18:03:59.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.209+0000 7f94abe00640 1 -- 172.21.15.82:0/2366099099 <== mon.1 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a002f990 con 0x7f94ac15c120 2024-03-16T18:03:59.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.209+0000 7f94abe00640 1 -- 172.21.15.82:0/2366099099 <== mon.1 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a002fd20 con 0x7f94ac15c120 2024-03-16T18:03:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.209+0000 7f94b3e00640 1 -- 172.21.15.82:0/2366099099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94ac15c120 msgr2=0x7f94ac0cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.209+0000 7f94b3e00640 1 --2- 172.21.15.82:0/2366099099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94ac15c120 0x7f94ac0cd7e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94a00098a0 tx=0x7f94a002ba10 comp rx=0 tx=0).stop 2024-03-16T18:03:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.210+0000 7f94b3e00640 1 -- 172.21.15.82:0/2366099099 shutdown_connections 2024-03-16T18:03:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.210+0000 7f94b3e00640 1 --2- 172.21.15.82:0/2366099099 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0d08b0 0x7f94ac0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.210+0000 7f94b3e00640 1 --2- 172.21.15.82:0/2366099099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94ac15c120 0x7f94ac0cd7e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.210+0000 7f94b3e00640 1 -- 172.21.15.82:0/2366099099 >> 172.21.15.82:0/2366099099 conn(0x7f94ac0b3710 msgr2=0x7f94ac0b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:59.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.210+0000 7f94b3e00640 1 -- 172.21.15.82:0/2366099099 shutdown_connections 2024-03-16T18:03:59.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.210+0000 7f94b3e00640 1 -- 172.21.15.82:0/2366099099 wait complete. 2024-03-16T18:03:59.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.211+0000 7f94b3e00640 1 Processor -- start 2024-03-16T18:03:59.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.212+0000 7f94b3e00640 1 -- start start 2024-03-16T18:03:59.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.212+0000 7f94b3e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0cf150 0x7f94ac0cb660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b3e00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94ac0d08b0 0x7f94ac0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b3e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94ac15c120 0x7f94ac150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b3e00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0b6630 con 0x7f94ac15c120 2024-03-16T18:03:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b3e00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0b6910 con 0x7f94ac0d08b0 2024-03-16T18:03:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b3e00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac0b6bf0 con 0x7f94ac0cf150 2024-03-16T18:03:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94ac15c120 0x7f94ac150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94ac15c120 0x7f94ac150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60286/0 (socket says 172.21.15.82:60286) 2024-03-16T18:03:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b1e00640 1 -- 172.21.15.82:0/1527275314 learned_addr learned my addr 172.21.15.82:0/1527275314 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b0a00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94ac0d08b0 0x7f94ac0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b1400640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0cf150 0x7f94ac0cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b1e00640 1 -- 172.21.15.82:0/1527275314 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0cf150 msgr2=0x7f94ac0cb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b1e00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0cf150 0x7f94ac0cb660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b1e00640 1 -- 172.21.15.82:0/1527275314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94ac0d08b0 msgr2=0x7f94ac0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.213+0000 7f94b1e00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94ac0d08b0 0x7f94ac0b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.214+0000 7f94b1e00640 1 -- 172.21.15.82:0/1527275314 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a4009a70 con 0x7f94ac15c120 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.214+0000 7f94b0a00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94ac0d08b0 0x7f94ac0b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.214+0000 7f94b1e00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94ac15c120 0x7f94ac150780 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f94a400efc0 tx=0x7f94a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.214+0000 7f94a9600640 1 -- 172.21.15.82:0/1527275314 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a400cd30 con 0x7f94ac15c120 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.214+0000 7f94a9600640 1 -- 172.21.15.82:0/1527275314 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a400f040 con 0x7f94ac15c120 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.214+0000 7f94a9600640 1 -- 172.21.15.82:0/1527275314 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4003bd0 con 0x7f94ac15c120 2024-03-16T18:03:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.214+0000 7f94b3e00640 1 -- 172.21.15.82:0/1527275314 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a00094d0 con 0x7f94ac15c120 2024-03-16T18:03:59.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.214+0000 7f94b3e00640 1 -- 172.21.15.82:0/1527275314 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ac0c9620 con 0x7f94ac15c120 2024-03-16T18:03:59.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.215+0000 7f94b3e00640 1 -- 172.21.15.82:0/1527275314 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ac0690e0 con 0x7f94ac15c120 2024-03-16T18:03:59.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.217+0000 7f94a9600640 1 -- 172.21.15.82:0/1527275314 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f94a4014480 con 0x7f94ac15c120 2024-03-16T18:03:59.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.217+0000 7f94a9600640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f948c06b3b0 0x7f948c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.218+0000 7f94a9600640 1 -- 172.21.15.82:0/1527275314 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f94a408a440 con 0x7f94ac15c120 2024-03-16T18:03:59.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.218+0000 7f94b1400640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f948c06b3b0 0x7f948c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.219+0000 7f94b1400640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f948c06b3b0 0x7f948c06d870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f94a0036000 tx=0x7f94a0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:59.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.222+0000 7f94a9600640 1 -- 172.21.15.82:0/1527275314 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f94a4055430 con 0x7f94ac15c120 2024-03-16T18:03:59.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.418+0000 7f94b3e00640 1 -- 172.21.15.82:0/1527275314 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f94ac0cd2c0 con 0x7f948c06b3b0 2024-03-16T18:03:59.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.419+0000 7f94a9600640 1 -- 172.21.15.82:0/1527275314 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f94a002fb00 con 0x7f948c06b3b0 2024-03-16T18:03:59.422 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 6 to deep-scrub 2024-03-16T18:03:59.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.421+0000 7f94b3e00640 1 -- 172.21.15.82:0/1527275314 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f948c06b3b0 msgr2=0x7f948c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f948c06b3b0 0x7f948c06d870 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f94a0036000 tx=0x7f94a0008030 comp rx=0 tx=0).stop 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 -- 172.21.15.82:0/1527275314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94ac15c120 msgr2=0x7f94ac150780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94ac15c120 0x7f94ac150780 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f94a400efc0 tx=0x7f94a400c490 comp rx=0 tx=0).stop 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 -- 172.21.15.82:0/1527275314 shutdown_connections 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f948c06b3b0 0x7f948c06d870 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94ac15c120 0x7f94ac150780 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94ac0d08b0 0x7f94ac0b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 --2- 172.21.15.82:0/1527275314 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94ac0cf150 0x7f94ac0cb660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 -- 172.21.15.82:0/1527275314 >> 172.21.15.82:0/1527275314 conn(0x7f94ac0b3710 msgr2=0x7f94ac15c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 -- 172.21.15.82:0/1527275314 shutdown_connections 2024-03-16T18:03:59.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.422+0000 7f94b3e00640 1 -- 172.21.15.82:0/1527275314 wait complete. 2024-03-16T18:03:59.444 INFO:tasks.ceph:Scrubbing osd.7 2024-03-16T18:03:59.444 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-16T18:03:59.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.654+0000 7f7aeb800640 1 Processor -- start 2024-03-16T18:03:59.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.654+0000 7f7aeb800640 1 -- start start 2024-03-16T18:03:59.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.656+0000 7f7aeb800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 0x7f7ae40cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.656+0000 7f7aeb800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7ae415e310 0x7f7ae40d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.656+0000 7f7aeb800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae406ee70 con 0x7f7ae40ce4f0 2024-03-16T18:03:59.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.656+0000 7f7aeb800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae4067ac0 con 0x7f7ae415e310 2024-03-16T18:03:59.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.656+0000 7f7aeb800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae406f150 con 0x7f7ae415df30 2024-03-16T18:03:59.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.656+0000 7f7ae3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7ae415e310 0x7f7ae40d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.656+0000 7f7ae8e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 0x7f7ae40cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.657+0000 7f7ae3e00640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7f7ae415e310 0x7f7ae40d0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.82:55078/0 (socket says 172.21.15.82:55078) 2024-03-16T18:03:59.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.657+0000 7f7ae3e00640 1 -- 172.21.15.82:0/4015446157 learned_addr learned my addr 172.21.15.82:0/4015446157 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:59.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.657+0000 7f7ae8e00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 0x7f7ae40cfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3301/0 says I am v2:172.21.15.82:40342/0 (socket says 172.21.15.82:40342) 2024-03-16T18:03:59.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.657+0000 7f7ae8e00640 1 -- 172.21.15.82:0/4015446157 >> v2:172.21.15.90:3300/0 conn(0x7f7ae415e310 msgr2=0x7f7ae40d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:03:59.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.657+0000 7f7ae8e00640 1 --2- 172.21.15.82:0/4015446157 >> v2:172.21.15.90:3300/0 conn(0x7f7ae415e310 0x7f7ae40d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.657+0000 7f7ae8e00640 1 -- 172.21.15.82:0/4015446157 >> v1:172.21.15.90:6789/0 conn(0x7f7ae40ce4f0 legacy=0x7f7ae4161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.657+0000 7f7ae8e00640 1 -- 172.21.15.82:0/4015446157 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae406f430 con 0x7f7ae415df30 2024-03-16T18:03:59.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.657+0000 7f7ae3e00640 1 --2- 172.21.15.82:0/4015446157 >> v2:172.21.15.90:3300/0 conn(0x7f7ae415e310 0x7f7ae40d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T18:03:59.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.659+0000 7f7ae8e00640 1 --2- 172.21.15.82:0/4015446157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 0x7f7ae40cfeb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40097a0 tx=0x7f7ad402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f36e9142f24f1234 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:59.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.659+0000 7f7ae3400640 1 -- 172.21.15.82:0/4015446157 <== mon.2 v2:172.21.15.90:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad402d040 con 0x7f7ae415df30 2024-03-16T18:03:59.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.659+0000 7f7ae3400640 1 -- 172.21.15.82:0/4015446157 <== mon.2 v2:172.21.15.90:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad4031810 con 0x7f7ae415df30 2024-03-16T18:03:59.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.659+0000 7f7ae3400640 1 -- 172.21.15.82:0/4015446157 <== mon.2 v2:172.21.15.90:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4031ba0 con 0x7f7ae415df30 2024-03-16T18:03:59.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.660+0000 7f7aeb800640 1 -- 172.21.15.82:0/4015446157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 msgr2=0x7f7ae40cfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.660+0000 7f7aeb800640 1 --2- 172.21.15.82:0/4015446157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 0x7f7ae40cfeb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40097a0 tx=0x7f7ad402b8b0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.660+0000 7f7aeb800640 1 -- 172.21.15.82:0/4015446157 shutdown_connections 2024-03-16T18:03:59.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.660+0000 7f7aeb800640 1 --2- 172.21.15.82:0/4015446157 >> v2:172.21.15.90:3300/0 conn(0x7f7ae415e310 0x7f7ae40d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.660+0000 7f7aeb800640 1 --2- 172.21.15.82:0/4015446157 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 0x7f7ae40cfeb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.660+0000 7f7aeb800640 1 -- 172.21.15.82:0/4015446157 >> 172.21.15.82:0/4015446157 conn(0x7f7ae40d1c70 msgr2=0x7f7ae40d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:59.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.660+0000 7f7aeb800640 1 -- 172.21.15.82:0/4015446157 shutdown_connections 2024-03-16T18:03:59.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.661+0000 7f7aeb800640 1 -- 172.21.15.82:0/4015446157 wait complete. 2024-03-16T18:03:59.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.663+0000 7f7aeb800640 1 Processor -- start 2024-03-16T18:03:59.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.663+0000 7f7aeb800640 1 -- start start 2024-03-16T18:03:59.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.664+0000 7f7aeb800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ae40ce4f0 0x7f7ae4159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.664+0000 7f7aeb800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 0x7f7ae40c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.664+0000 7f7aeb800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae415e310 0x7f7ae40c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.664+0000 7f7aeb800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40cbd30 con 0x7f7ae415e310 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.664+0000 7f7aeb800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40cc010 con 0x7f7ae40ce4f0 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.664+0000 7f7aeb800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40cc2f0 con 0x7f7ae415df30 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae415e310 0x7f7ae40c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae9800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae415e310 0x7f7ae40c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60296/0 (socket says 172.21.15.82:60296) 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae9800640 1 -- 172.21.15.82:0/4091245046 learned_addr learned my addr 172.21.15.82:0/4091245046 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae3e00640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 0x7f7ae40c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae8e00640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ae40ce4f0 0x7f7ae4159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae9800640 1 -- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 msgr2=0x7f7ae40c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae9800640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f7ae415df30 0x7f7ae40c8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae9800640 1 -- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ae40ce4f0 msgr2=0x7f7ae4159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae9800640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ae40ce4f0 0x7f7ae4159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae9800640 1 -- 172.21.15.82:0/4091245046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc009b30 con 0x7f7ae415e310 2024-03-16T18:03:59.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.665+0000 7f7ae8e00640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ae40ce4f0 0x7f7ae4159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T18:03:59.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.666+0000 7f7ae9800640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae415e310 0x7f7ae40c9280 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00c390 tx=0x7f7adc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:59.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.666+0000 7f7ae0c00640 1 -- 172.21.15.82:0/4091245046 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0052d0 con 0x7f7ae415e310 2024-03-16T18:03:59.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.666+0000 7f7ae0c00640 1 -- 172.21.15.82:0/4091245046 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc005430 con 0x7f7ae415e310 2024-03-16T18:03:59.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.666+0000 7f7ae0c00640 1 -- 172.21.15.82:0/4091245046 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc003810 con 0x7f7ae415e310 2024-03-16T18:03:59.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.666+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad4009400 con 0x7f7ae415e310 2024-03-16T18:03:59.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.666+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae4070090 con 0x7f7ae415e310 2024-03-16T18:03:59.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.667+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7ae40cbea0 con 0x7f7ae415e310 2024-03-16T18:03:59.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.668+0000 7f7ae0c00640 1 -- 172.21.15.82:0/4091245046 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f7adc003970 con 0x7f7ae415e310 2024-03-16T18:03:59.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.669+0000 7f7ae0c00640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7ac006b3b0 0x7f7ac006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.669+0000 7f7ae0c00640 1 -- 172.21.15.82:0/4091245046 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7adc088e20 con 0x7f7ae415e310 2024-03-16T18:03:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.669+0000 7f7ae0c00640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] conn(0x7f7ac0071270 0x7f7ac0073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.669+0000 7f7ae8e00640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7ac006b3b0 0x7f7ac006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.669+0000 7f7ae0c00640 1 -- 172.21.15.82:0/4091245046 --> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ac0073d40 con 0x7f7ac0071270 2024-03-16T18:03:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.669+0000 7f7ae0c00640 1 -- 172.21.15.82:0/4091245046 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7adc08b400 con 0x7f7ae415e310 2024-03-16T18:03:59.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.670+0000 7f7ae3e00640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] conn(0x7f7ac0071270 0x7f7ac0073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.670+0000 7f7ae8e00640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7ac006b3b0 0x7f7ac006d870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4036000 tx=0x7f7ad403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:59.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.671+0000 7f7ae3e00640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] conn(0x7f7ac0071270 0x7f7ac0073690 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:59.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.673+0000 7f7ae0c00640 1 -- 172.21.15.82:0/4091245046 <== osd.7 v2:172.21.15.90:6817/3392471023 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7ac0073d40 con 0x7f7ac0071270 2024-03-16T18:03:59.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.689+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 --> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7ae40cc180 con 0x7f7ac0071270 2024-03-16T18:03:59.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.690+0000 7f7ae0c00640 1 -- 172.21.15.82:0/4091245046 <== osd.7 v2:172.21.15.90:6817/3392471023 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7ae40cc180 con 0x7f7ac0071270 2024-03-16T18:03:59.694 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-03-16T18:03:59.694 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T18:03:59.694 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.690+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] conn(0x7f7ac0071270 msgr2=0x7f7ac0073690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.690+0000 7f7aeb800640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.90:6817/3392471023,v1:172.21.15.90:6819/3392471023] conn(0x7f7ac0071270 0x7f7ac0073690 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.691+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7ac006b3b0 msgr2=0x7f7ac006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.691+0000 7f7aeb800640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f7ac006b3b0 0x7f7ac006d870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4036000 tx=0x7f7ad403c040 comp rx=0 tx=0).stop 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.691+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae415e310 msgr2=0x7f7ae40c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.691+0000 7f7aeb800640 1 --2- 172.21.15.82:0/4091245046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae415e310 0x7f7ae40c9280 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00c390 tx=0x7f7adc00c850 comp rx=0 tx=0).stop 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.691+0000 7f7ae9800640 1 -- 172.21.15.82:0/4091245046 reap_dead start 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.691+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 shutdown_connections 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.691+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 >> 172.21.15.82:0/4091245046 conn(0x7f7ae40d1c70 msgr2=0x7f7ae415a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.691+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 shutdown_connections 2024-03-16T18:03:59.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.691+0000 7f7aeb800640 1 -- 172.21.15.82:0/4091245046 wait complete. 2024-03-16T18:03:59.713 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-16T18:03:59.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.916+0000 7fb630800640 1 Processor -- start 2024-03-16T18:03:59.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.916+0000 7fb630800640 1 -- start start 2024-03-16T18:03:59.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.918+0000 7fb630800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6280aec80 0x7fb6280ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.918+0000 7fb630800640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62806ded0 con 0x7fb6281450e0 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.918+0000 7fb630800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6280af060 con 0x7fb6280ce850 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.918+0000 7fb630800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb628072c10 con 0x7fb6280aec80 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.918+0000 7fb62de00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6280aec80 0x7fb6280ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.918+0000 7fb62e800640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb6281450e0 0x7fb62815f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59036/0 (socket says 172.21.15.82:59036) 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.918+0000 7fb62de00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6280aec80 0x7fb6280ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60306/0 (socket says 172.21.15.82:60306) 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.918+0000 7fb62e800640 1 -- 172.21.15.82:0/1339209400 learned_addr learned my addr 172.21.15.82:0/1339209400 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.919+0000 7fb62de00640 1 -- 172.21.15.82:0/1339209400 >> v1:172.21.15.90:6789/0 conn(0x7fb6280ce850 legacy=0x7fb628070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.919+0000 7fb62de00640 1 -- 172.21.15.82:0/1339209400 >> v1:172.21.15.82:6789/0 conn(0x7fb6281450e0 legacy=0x7fb62815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.919+0000 7fb62ca00640 1 -- 172.21.15.82:0/1339209400 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984205367 0 0) 0x7fb624001490 con 0x7fb6281450e0 2024-03-16T18:03:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.919+0000 7fb62de00640 1 -- 172.21.15.82:0/1339209400 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb628160730 con 0x7fb6280aec80 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.920+0000 7fb62de00640 1 --2- 172.21.15.82:0/1339209400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280aec80 0x7fb6280ce310 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0097a0 tx=0x7fb61c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af8ba00131527e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.920+0000 7fb62ca00640 1 -- 172.21.15.82:0/1339209400 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c031830 con 0x7fb6280aec80 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.920+0000 7fb62ca00640 1 -- 172.21.15.82:0/1339209400 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c031990 con 0x7fb6280aec80 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.920+0000 7fb62ca00640 1 -- 172.21.15.82:0/1339209400 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c031d20 con 0x7fb6280aec80 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.921+0000 7fb630800640 1 -- 172.21.15.82:0/1339209400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280aec80 msgr2=0x7fb6280ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.921+0000 7fb630800640 1 --2- 172.21.15.82:0/1339209400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280aec80 0x7fb6280ce310 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb61c0097a0 tx=0x7fb61c02b8d0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.921+0000 7fb630800640 1 -- 172.21.15.82:0/1339209400 shutdown_connections 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.921+0000 7fb630800640 1 --2- 172.21.15.82:0/1339209400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280aec80 0x7fb6280ce310 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.921+0000 7fb630800640 1 -- 172.21.15.82:0/1339209400 >> 172.21.15.82:0/1339209400 conn(0x7fb6280b17b0 msgr2=0x7fb6280b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.921+0000 7fb630800640 1 -- 172.21.15.82:0/1339209400 shutdown_connections 2024-03-16T18:03:59.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.921+0000 7fb630800640 1 -- 172.21.15.82:0/1339209400 wait complete. 2024-03-16T18:03:59.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.923+0000 7fb630800640 1 Processor -- start 2024-03-16T18:03:59.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.923+0000 7fb630800640 1 -- start start 2024-03-16T18:03:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.924+0000 7fb630800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6280aec80 0x7fb6280b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.924+0000 7fb630800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb6281450e0 0x7fb6280b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.924+0000 7fb630800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280b49e0 0x7fb6280b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.924+0000 7fb630800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280b3400 con 0x7fb6280b49e0 2024-03-16T18:03:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.924+0000 7fb630800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280b36e0 con 0x7fb6280aec80 2024-03-16T18:03:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.924+0000 7fb630800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6280b39c0 con 0x7fb6281450e0 2024-03-16T18:03:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.924+0000 7fb62e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280b49e0 0x7fb6280b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.924+0000 7fb62e800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280b49e0 0x7fb6280b02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60308/0 (socket says 172.21.15.82:60308) 2024-03-16T18:03:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.924+0000 7fb62e800640 1 -- 172.21.15.82:0/1326208397 learned_addr learned my addr 172.21.15.82:0/1326208397 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:03:59.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb62de00640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6280aec80 0x7fb6280b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb62d400640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb6281450e0 0x7fb6280b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb62e800640 1 -- 172.21.15.82:0/1326208397 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb6281450e0 msgr2=0x7fb6280b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb62e800640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb6281450e0 0x7fb6280b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb62e800640 1 -- 172.21.15.82:0/1326208397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6280aec80 msgr2=0x7fb6280b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb62e800640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6280aec80 0x7fb6280b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb62e800640 1 -- 172.21.15.82:0/1326208397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c009400 con 0x7fb6280b49e0 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb62de00640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6280aec80 0x7fb6280b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb62e800640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280b49e0 0x7fb6280b02b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c7f0 tx=0x7fb62400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb622000640 1 -- 172.21.15.82:0/1326208397 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62400a5e0 con 0x7fb6280b49e0 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb622000640 1 -- 172.21.15.82:0/1326208397 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62400a760 con 0x7fb6280b49e0 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb622000640 1 -- 172.21.15.82:0/1326208397 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62400ab50 con 0x7fb6280b49e0 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb630800640 1 -- 172.21.15.82:0/1326208397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6280b3ca0 con 0x7fb6280b49e0 2024-03-16T18:03:59.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.925+0000 7fb630800640 1 -- 172.21.15.82:0/1326208397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6280b30f0 con 0x7fb6280b49e0 2024-03-16T18:03:59.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.927+0000 7fb630800640 1 -- 172.21.15.82:0/1326208397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6280b41c0 con 0x7fb6280b49e0 2024-03-16T18:03:59.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.927+0000 7fb622000640 1 -- 172.21.15.82:0/1326208397 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fb62400acb0 con 0x7fb6280b49e0 2024-03-16T18:03:59.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.929+0000 7fb622000640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb60c06b3b0 0x7fb60c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:03:59.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.929+0000 7fb622000640 1 -- 172.21.15.82:0/1326208397 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb624088f40 con 0x7fb6280b49e0 2024-03-16T18:03:59.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.929+0000 7fb62de00640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb60c06b3b0 0x7fb60c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:03:59.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.930+0000 7fb62de00640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb60c06b3b0 0x7fb60c06d870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb61c036000 tx=0x7fb61c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:03:59.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:03:59.934+0000 7fb622000640 1 -- 172.21.15.82:0/1326208397 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb6280b41c0 con 0x7fb6280b49e0 2024-03-16T18:04:00.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.126+0000 7fb630800640 1 -- 172.21.15.82:0/1326208397 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fb628072620 con 0x7fb60c06b3b0 2024-03-16T18:04:00.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.126+0000 7fb622000640 1 -- 172.21.15.82:0/1326208397 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb61c03a420 con 0x7fb60c06b3b0 2024-03-16T18:04:00.130 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 7 to deep-scrub 2024-03-16T18:04:00.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 -- 172.21.15.82:0/1326208397 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb60c06b3b0 msgr2=0x7fb60c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb60c06b3b0 0x7fb60c06d870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb61c036000 tx=0x7fb61c03c040 comp rx=0 tx=0).stop 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 -- 172.21.15.82:0/1326208397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280b49e0 msgr2=0x7fb6280b02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280b49e0 0x7fb6280b02b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c7f0 tx=0x7fb62400ccb0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 -- 172.21.15.82:0/1326208397 shutdown_connections 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fb60c06b3b0 0x7fb60c06d870 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6280b49e0 0x7fb6280b02b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fb6281450e0 0x7fb6280b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 --2- 172.21.15.82:0/1326208397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6280aec80 0x7fb6280b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 -- 172.21.15.82:0/1326208397 >> 172.21.15.82:0/1326208397 conn(0x7fb6280b17b0 msgr2=0x7fb628070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 -- 172.21.15.82:0/1326208397 shutdown_connections 2024-03-16T18:04:00.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.129+0000 7fb630800640 1 -- 172.21.15.82:0/1326208397 wait complete. 2024-03-16T18:04:00.151 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T18:04:00.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.348+0000 7f944a400640 1 Processor -- start 2024-03-16T18:04:00.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.349+0000 7f944a400640 1 -- start start 2024-03-16T18:04:00.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.350+0000 7f944a400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9444143e00 0x7f9444070c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:00.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.350+0000 7f944a400640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94440cf3a0 con 0x7f9444070820 2024-03-16T18:04:00.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.350+0000 7f944a400640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9444071140 con 0x7f94440cd660 2024-03-16T18:04:00.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.350+0000 7f944a400640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94440cf610 con 0x7f9444143e00 2024-03-16T18:04:00.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.350+0000 7f9449000640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9444070820 0x7f944415f080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59042/0 (socket says 172.21.15.82:59042) 2024-03-16T18:04:00.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.350+0000 7f9449000640 1 -- 172.21.15.82:0/2208093389 learned_addr learned my addr 172.21.15.82:0/2208093389 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:04:00.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.350+0000 7f9443e00640 1 --2- 172.21.15.82:0/2208093389 >> v2:172.21.15.82:3300/0 conn(0x7f9444143e00 0x7f9444070c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:00.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.351+0000 7f9443e00640 1 -- 172.21.15.82:0/2208093389 >> v1:172.21.15.90:6789/0 conn(0x7f94440cd660 legacy=0x7f94440cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:04:00.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.351+0000 7f9443e00640 1 -- 172.21.15.82:0/2208093389 >> v1:172.21.15.82:6789/0 conn(0x7f9444070820 legacy=0x7f944415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:00.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.351+0000 7f9443e00640 1 -- 172.21.15.82:0/2208093389 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94440cf8f0 con 0x7f9444143e00 2024-03-16T18:04:00.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.352+0000 7f9443e00640 1 --2- 172.21.15.82:0/2208093389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 0x7f9444070c00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f94300097a0 tx=0x7f943002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b0c600186da42a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:00.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.352+0000 7f9443400640 1 -- 172.21.15.82:0/2208093389 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943002e070 con 0x7f9444143e00 2024-03-16T18:04:00.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.352+0000 7f9443400640 1 -- 172.21.15.82:0/2208093389 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94300315a0 con 0x7f9444143e00 2024-03-16T18:04:00.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.352+0000 7f9443400640 1 -- 172.21.15.82:0/2208093389 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9430031930 con 0x7f9444143e00 2024-03-16T18:04:00.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.353+0000 7f944a400640 1 -- 172.21.15.82:0/2208093389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 msgr2=0x7f9444070c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:00.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.353+0000 7f944a400640 1 --2- 172.21.15.82:0/2208093389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 0x7f9444070c00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f94300097a0 tx=0x7f943002b840 comp rx=0 tx=0).stop 2024-03-16T18:04:00.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.353+0000 7f944a400640 1 -- 172.21.15.82:0/2208093389 shutdown_connections 2024-03-16T18:04:00.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.353+0000 7f944a400640 1 --2- 172.21.15.82:0/2208093389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 0x7f9444070c00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.353+0000 7f944a400640 1 -- 172.21.15.82:0/2208093389 >> 172.21.15.82:0/2208093389 conn(0x7f944406ae70 msgr2=0x7f944406b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:04:00.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.353+0000 7f944a400640 1 -- 172.21.15.82:0/2208093389 shutdown_connections 2024-03-16T18:04:00.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.353+0000 7f944a400640 1 -- 172.21.15.82:0/2208093389 wait complete. 2024-03-16T18:04:00.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.355+0000 7f944a400640 1 Processor -- start 2024-03-16T18:04:00.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.355+0000 7f944a400640 1 -- start start 2024-03-16T18:04:00.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.356+0000 7f944a400640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9444070820 0x7f94440aad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:00.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.356+0000 7f944a400640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94440cd660 0x7f94441e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:00.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.356+0000 7f944a400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 0x7f94441f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:00.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.356+0000 7f944a400640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94440ab280 con 0x7f9444143e00 2024-03-16T18:04:00.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.356+0000 7f944a400640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94441e0ed0 con 0x7f9444070820 2024-03-16T18:04:00.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.356+0000 7f944a400640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94441e11b0 con 0x7f94440cd660 2024-03-16T18:04:00.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9449a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 0x7f94441f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:00.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9449a00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 0x7f94441f2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60334/0 (socket says 172.21.15.82:60334) 2024-03-16T18:04:00.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9449a00640 1 -- 172.21.15.82:0/784372433 learned_addr learned my addr 172.21.15.82:0/784372433 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:04:00.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9449000640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9444070820 0x7f94440aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:00.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9443e00640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94440cd660 0x7f94441e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:00.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9449a00640 1 -- 172.21.15.82:0/784372433 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94440cd660 msgr2=0x7f94441e1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:00.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9449a00640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94440cd660 0x7f94441e1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9449a00640 1 -- 172.21.15.82:0/784372433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9444070820 msgr2=0x7f94440aad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:00.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9449a00640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9444070820 0x7f94440aad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9449a00640 1 -- 172.21.15.82:0/784372433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9430009400 con 0x7f9444143e00 2024-03-16T18:04:00.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.357+0000 7f9443e00640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94440cd660 0x7f94441e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T18:04:00.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.358+0000 7f9449a00640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 0x7f94441f2df0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c780 tx=0x7f943c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:00.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.358+0000 7f9440c00640 1 -- 172.21.15.82:0/784372433 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c004b60 con 0x7f9444143e00 2024-03-16T18:04:00.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.358+0000 7f944a400640 1 -- 172.21.15.82:0/784372433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94441e1490 con 0x7f9444143e00 2024-03-16T18:04:00.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.358+0000 7f944a400640 1 -- 172.21.15.82:0/784372433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94441e1a50 con 0x7f9444143e00 2024-03-16T18:04:00.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.358+0000 7f9440c00640 1 -- 172.21.15.82:0/784372433 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c00ce90 con 0x7f9444143e00 2024-03-16T18:04:00.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.358+0000 7f9440c00640 1 -- 172.21.15.82:0/784372433 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c006b50 con 0x7f9444143e00 2024-03-16T18:04:00.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.360+0000 7f944a400640 1 -- 172.21.15.82:0/784372433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9420005290 con 0x7f9444143e00 2024-03-16T18:04:00.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.361+0000 7f9440c00640 1 -- 172.21.15.82:0/784372433 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7f943c00a850 con 0x7f9444143e00 2024-03-16T18:04:00.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.361+0000 7f9440c00640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f942406b3b0 0x7f942406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:00.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.362+0000 7f9440c00640 1 -- 172.21.15.82:0/784372433 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f943c0898a0 con 0x7f9444143e00 2024-03-16T18:04:00.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.362+0000 7f9449000640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f942406b3b0 0x7f942406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:00.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.363+0000 7f9449000640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f942406b3b0 0x7f942406d870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9438000f60 tx=0x7f9438009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:00.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.367+0000 7f9440c00640 1 -- 172.21.15.82:0/784372433 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f943c054780 con 0x7f9444143e00 2024-03-16T18:04:00.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.547+0000 7f944a400640 1 -- 172.21.15.82:0/784372433 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9420000bc0 con 0x7f942406b3b0 2024-03-16T18:04:00.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.549+0000 7f9440c00640 1 -- 172.21.15.82:0/784372433 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13351 (secure 0 0 0) 0x7f9420000bc0 con 0x7f942406b3b0 2024-03-16T18:04:00.553 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:04:00.553 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 -- 172.21.15.82:0/784372433 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f942406b3b0 msgr2=0x7f942406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f942406b3b0 0x7f942406d870 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9438000f60 tx=0x7f9438009040 comp rx=0 tx=0).stop 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 -- 172.21.15.82:0/784372433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 msgr2=0x7f94441f2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 0x7f94441f2df0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c780 tx=0x7f943c00cc40 comp rx=0 tx=0).stop 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 -- 172.21.15.82:0/784372433 shutdown_connections 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7f942406b3b0 0x7f942406d870 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9444143e00 0x7f94441f2df0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7f94440cd660 0x7f94441e1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 --2- 172.21.15.82:0/784372433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9444070820 0x7f94440aad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 -- 172.21.15.82:0/784372433 >> 172.21.15.82:0/784372433 conn(0x7f944406ae70 msgr2=0x7f94440ccb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 -- 172.21.15.82:0/784372433 shutdown_connections 2024-03-16T18:04:00.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:00.552+0000 7f944a400640 1 -- 172.21.15.82:0/784372433 wait complete. 2024-03-16T18:04:00.574 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-03-16T18:04:00.125422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220664,"kb_used_data":4488,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754056,"statfs":{"total":773094113280,"available":772868153344,"internally_reserved":0,"allocated":4595712,"data_stored":2521984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.831088"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-16T18:03:49.769304+0000","last_change":"2024-03-16T17:57:35.560172+0000","last_active":"2024-03-16T18:03:49.769304+0000","last_peered":"2024-03-16T18:03:49.769304+0000","last_clean":"2024-03-16T18:03:49.769304+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T18:03:49.769304+0000","last_undegraded":"2024-03-16T18:03:49.769304+0000","last_fullsized":"2024-03-16T18:03:49.769304+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T17:57:34.174437+0000","last_clean_scrub_stamp":"2024-03-16T17:57:34.174437+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:20:03.039404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542225,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":758024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542225,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542223,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542223,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542223,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542224,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542225,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":758024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T18:04:00.577 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-16T17:57:34.174437+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=16, tm_hour=17, tm_min=57, tm_sec=34, tm_wday=5, tm_yday=76, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=16, tm_hour=18, tm_min=3, tm_sec=53, tm_wday=5, tm_yday=76, tm_isdst=0) 2024-03-16T18:04:00.577 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-16T18:04:07.356 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:04:07.364 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:04:20.578 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T18:04:20.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.795+0000 7fe41ec00640 1 Processor -- start 2024-03-16T18:04:20.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.796+0000 7fe41ec00640 1 -- start start 2024-03-16T18:04:20.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.797+0000 7fe41ec00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180ae9d0 0x7fe4180b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:20.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.797+0000 7fe41ec00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4180b18d0 con 0x7fe41806c970 2024-03-16T18:04:20.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.797+0000 7fe41ec00640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4180b1bb0 con 0x7fe4180b6a30 2024-03-16T18:04:20.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.797+0000 7fe41ec00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180b1e90 con 0x7fe4180ae9d0 2024-03-16T18:04:20.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.797+0000 7fe417e00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe41806c970 0x7fe41806cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48412/0 (socket says 172.21.15.82:48412) 2024-03-16T18:04:20.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.797+0000 7fe417e00640 1 -- 172.21.15.82:0/3400883675 learned_addr learned my addr 172.21.15.82:0/3400883675 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:04:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.798+0000 7fe417400640 1 --2- 172.21.15.82:0/3400883675 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180ae9d0 0x7fe4180b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.798+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1480371644 0 0) 0x7fe4180b18d0 con 0x7fe41806c970 2024-03-16T18:04:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.798+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3fc003520 con 0x7fe41806c970 2024-03-16T18:04:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.798+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2315632560 0 0) 0x7fe3fc003520 con 0x7fe41806c970 2024-03-16T18:04:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.798+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180ae9d0 msgr2=0x7fe4180b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:04:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.798+0000 7fe416a00640 1 --2- 172.21.15.82:0/3400883675 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180ae9d0 0x7fe4180b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.798+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 >> v1:172.21.15.90:6789/0 conn(0x7fe4180b6a30 legacy=0x7fe4180b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:20.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.798+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180b2170 con 0x7fe41806c970 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3316644474 0 0) 0x7fe4080019f0 con 0x7fe41806c970 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 >> v1:172.21.15.82:6789/0 conn(0x7fe41806c970 legacy=0x7fe41806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe416a00640 1 --2- 172.21.15.82:0/3400883675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3fc0039a0 0x7fe3fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe416a00640 1 --2- 172.21.15.82:0/3400883675 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180ae9d0 0x7fe3fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe416a00640 1 --2- 172.21.15.82:0/3400883675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc004d00 0x7fe3fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180b1e90 con 0x7fe3fc004d00 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc00bac0 con 0x7fe3fc0039a0 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc00bc80 con 0x7fe4180ae9d0 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe41cc00640 1 --2- 172.21.15.82:0/3400883675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc004d00 0x7fe3fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe41cc00640 1 -- 172.21.15.82:0/3400883675 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180ae9d0 msgr2=0x7fe3fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe41cc00640 1 --2- 172.21.15.82:0/3400883675 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180ae9d0 0x7fe3fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe41cc00640 1 -- 172.21.15.82:0/3400883675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3fc0039a0 msgr2=0x7fe3fc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe41cc00640 1 --2- 172.21.15.82:0/3400883675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe3fc0039a0 0x7fe3fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:20.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.799+0000 7fe41cc00640 1 -- 172.21.15.82:0/3400883675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc003520 con 0x7fe3fc004d00 2024-03-16T18:04:20.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.800+0000 7fe41cc00640 1 --2- 172.21.15.82:0/3400883675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc004d00 0x7fe3fc005180 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe410001720 tx=0x7fe3fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba4539b944c70e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:20.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.801+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100052d0 con 0x7fe3fc004d00 2024-03-16T18:04:20.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.801+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41002e050 con 0x7fe3fc004d00 2024-03-16T18:04:20.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.801+0000 7fe416a00640 1 -- 172.21.15.82:0/3400883675 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100035f0 con 0x7fe3fc004d00 2024-03-16T18:04:20.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.801+0000 7fe41ec00640 1 -- 172.21.15.82:0/3400883675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc004d00 msgr2=0x7fe3fc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:20.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.801+0000 7fe41ec00640 1 --2- 172.21.15.82:0/3400883675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3fc004d00 0x7fe3fc005180 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fe410001720 tx=0x7fe3fc0057f0 comp rx=0 tx=0).stop 2024-03-16T18:04:20.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.801+0000 7fe41cc00640 1 -- 172.21.15.82:0/3400883675 reap_dead start 2024-03-16T18:04:20.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.801+0000 7fe41ec00640 1 -- 172.21.15.82:0/3400883675 shutdown_connections 2024-03-16T18:04:20.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.801+0000 7fe41ec00640 1 -- 172.21.15.82:0/3400883675 >> 172.21.15.82:0/3400883675 conn(0x7fe41809a9e0 msgr2=0x7fe41809adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:04:20.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.801+0000 7fe41ec00640 1 -- 172.21.15.82:0/3400883675 shutdown_connections 2024-03-16T18:04:20.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.801+0000 7fe41ec00640 1 -- 172.21.15.82:0/3400883675 wait complete. 2024-03-16T18:04:20.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.803+0000 7fe41ec00640 1 Processor -- start 2024-03-16T18:04:20.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.803+0000 7fe41ec00640 1 -- start start 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.804+0000 7fe41ec00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe418154300 0x7fe4180b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.804+0000 7fe41ec00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4180cb9f0 0x7fe4180cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.804+0000 7fe41ec00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180cd5b0 0x7fe418159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.804+0000 7fe41ec00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180cc060 con 0x7fe418154300 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.804+0000 7fe41ec00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180cc2b0 con 0x7fe4180cb9f0 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.804+0000 7fe41ec00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180cc590 con 0x7fe4180cd5b0 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.804+0000 7fe417e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe418154300 0x7fe4180b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.804+0000 7fe417e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe418154300 0x7fe4180b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34740/0 (socket says 172.21.15.82:34740) 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe417e00640 1 -- 172.21.15.82:0/3617533114 learned_addr learned my addr 172.21.15.82:0/3617533114 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe417e00640 1 -- 172.21.15.82:0/3617533114 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180cd5b0 msgr2=0x7fe418159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe417e00640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180cd5b0 0x7fe418159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:20.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe417e00640 1 -- 172.21.15.82:0/3617533114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4180cb9f0 msgr2=0x7fe4180cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:20.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe417e00640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4180cb9f0 0x7fe4180cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:20.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe417e00640 1 -- 172.21.15.82:0/3617533114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180cc870 con 0x7fe418154300 2024-03-16T18:04:20.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe417e00640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe418154300 0x7fe4180b4870 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe40800cb10 tx=0x7fe40800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:20.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe40fe00640 1 -- 172.21.15.82:0/3617533114 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40800abf0 con 0x7fe418154300 2024-03-16T18:04:20.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe40fe00640 1 -- 172.21.15.82:0/3617533114 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40800ad50 con 0x7fe418154300 2024-03-16T18:04:20.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe40fe00640 1 -- 172.21.15.82:0/3617533114 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408007310 con 0x7fe418154300 2024-03-16T18:04:20.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.805+0000 7fe41ec00640 1 -- 172.21.15.82:0/3617533114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4180ccb50 con 0x7fe418154300 2024-03-16T18:04:20.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.806+0000 7fe41ec00640 1 -- 172.21.15.82:0/3617533114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4180b4f90 con 0x7fe418154300 2024-03-16T18:04:20.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.807+0000 7fe41ec00640 1 -- 172.21.15.82:0/3617533114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4180ccd90 con 0x7fe418154300 2024-03-16T18:04:20.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.808+0000 7fe40fe00640 1 -- 172.21.15.82:0/3617533114 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fe408007470 con 0x7fe418154300 2024-03-16T18:04:20.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.809+0000 7fe40fe00640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3e0 0x7fe3f406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:20.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.809+0000 7fe40fe00640 1 -- 172.21.15.82:0/3617533114 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe408089790 con 0x7fe418154300 2024-03-16T18:04:20.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.809+0000 7fe417400640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3e0 0x7fe3f406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:20.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.810+0000 7fe417400640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3e0 0x7fe3f406d8a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe41809f7b0 tx=0x7fe400009500 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:20.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:20.814+0000 7fe40fe00640 1 -- 172.21.15.82:0/3617533114 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe4180ccd90 con 0x7fe418154300 2024-03-16T18:04:21.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.005+0000 7fe41ec00640 1 -- 172.21.15.82:0/3617533114 --> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4180ccd90 con 0x7fe3f406b3e0 2024-03-16T18:04:21.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.006+0000 7fe40fe00640 1 -- 172.21.15.82:0/3617533114 <== mgr.4103 v2:172.21.15.82:6832/3774193927 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13358 (secure 0 0 0) 0x7fe4180ccd90 con 0x7fe3f406b3e0 2024-03-16T18:04:21.010 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:04:21.010 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-03-16T18:04:21.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 -- 172.21.15.82:0/3617533114 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3e0 msgr2=0x7fe3f406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3e0 0x7fe3f406d8a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe41809f7b0 tx=0x7fe400009500 comp rx=0 tx=0).stop 2024-03-16T18:04:21.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 -- 172.21.15.82:0/3617533114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe418154300 msgr2=0x7fe4180b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe418154300 0x7fe4180b4870 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe40800cb10 tx=0x7fe40800a060 comp rx=0 tx=0).stop 2024-03-16T18:04:21.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 -- 172.21.15.82:0/3617533114 shutdown_connections 2024-03-16T18:04:21.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fe3f406b3e0 0x7fe3f406d8a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fe4180cd5b0 0x7fe418159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4180cb9f0 0x7fe4180cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 --2- 172.21.15.82:0/3617533114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe418154300 0x7fe4180b4870 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 -- 172.21.15.82:0/3617533114 >> 172.21.15.82:0/3617533114 conn(0x7fe41809a9e0 msgr2=0x7fe41806f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:04:21.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 -- 172.21.15.82:0/3617533114 shutdown_connections 2024-03-16T18:04:21.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.009+0000 7fe41ec00640 1 -- 172.21.15.82:0/3617533114 wait complete. 2024-03-16T18:04:21.030 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-03-16T18:04:20.131181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220656,"kb_used_data":4488,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754064,"statfs":{"total":773094113280,"available":772868161536,"internally_reserved":0,"allocated":4595712,"data_stored":2521984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003337"},"pg_stats":[{"pgid":"1.0","version":"37'119","reported_seq":213,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-03-16T18:04:00.859677+0000","last_change":"2024-03-16T18:04:00.859747+0000","last_active":"2024-03-16T18:04:00.859677+0000","last_peered":"2024-03-16T18:04:00.859677+0000","last_clean":"2024-03-16T18:04:00.859677+0000","last_became_active":"2024-03-16T17:57:35.557822+0000","last_became_peered":"2024-03-16T17:57:35.557822+0000","last_unstale":"2024-03-16T18:04:00.859677+0000","last_undegraded":"2024-03-16T18:04:00.859677+0000","last_fullsized":"2024-03-16T18:04:00.859677+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'119","last_scrub_stamp":"2024-03-16T18:04:00.859472+0000","last_deep_scrub":"37'119","last_deep_scrub_stamp":"2024-03-16T18:04:00.859472+0000","last_clean_scrub_stamp":"2024-03-16T18:04:00.859472+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:56:49.685475+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542229,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1019904,"data_stored":758024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542229,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542227,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542228,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27440,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":425984,"data_stored":167656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542229,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1019904,"data_stored":758024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T18:04:21.031 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-16T18:04:21.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.228+0000 7faeece00640 1 Processor -- start 2024-03-16T18:04:21.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.228+0000 7faeece00640 1 -- start start 2024-03-16T18:04:21.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.230+0000 7faeece00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faee806e310 0x7faee80cc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.230+0000 7faeece00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faee80b3bc0 0x7faee80b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.230+0000 7faeece00640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee80cc990 con 0x7faee81443c0 2024-03-16T18:04:21.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.230+0000 7faeece00640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee8070fa0 con 0x7faee80b3bc0 2024-03-16T18:04:21.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.230+0000 7faeece00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee80b4570 con 0x7faee806e310 2024-03-16T18:04:21.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.230+0000 7faee7400640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7faee81443c0 0x7faee8070890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48428/0 (socket says 172.21.15.82:48428) 2024-03-16T18:04:21.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.230+0000 7faee7400640 1 -- 172.21.15.82:0/100929493 learned_addr learned my addr 172.21.15.82:0/100929493 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-03-16T18:04:21.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.230+0000 7faee7e00640 1 --2- 172.21.15.82:0/100929493 >> v2:172.21.15.82:3300/0 conn(0x7faee80b3bc0 0x7faee80b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:21.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.231+0000 7faee6a00640 1 --2- 172.21.15.82:0/100929493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faee806e310 0x7faee80cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:21.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.231+0000 7faee7e00640 1 -- 172.21.15.82:0/100929493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faee806e310 msgr2=0x7faee80cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:04:21.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.231+0000 7faee7e00640 1 --2- 172.21.15.82:0/100929493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faee806e310 0x7faee80cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.231+0000 7faee7e00640 1 -- 172.21.15.82:0/100929493 >> v1:172.21.15.82:6789/0 conn(0x7faee81443c0 legacy=0x7faee8070890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.231+0000 7faee7e00640 1 -- 172.21.15.82:0/100929493 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee8072560 con 0x7faee80b3bc0 2024-03-16T18:04:21.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.232+0000 7faee7e00640 1 --2- 172.21.15.82:0/100929493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee80b3bc0 0x7faee80b4030 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faee0009d90 tx=0x7faee002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6f522b033d988d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:21.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.232+0000 7faee6000640 1 -- 172.21.15.82:0/100929493 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee002cc50 con 0x7faee80b3bc0 2024-03-16T18:04:21.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.232+0000 7faee6000640 1 -- 172.21.15.82:0/100929493 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee002cdb0 con 0x7faee80b3bc0 2024-03-16T18:04:21.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.232+0000 7faee6000640 1 -- 172.21.15.82:0/100929493 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0034660 con 0x7faee80b3bc0 2024-03-16T18:04:21.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.233+0000 7faeece00640 1 -- 172.21.15.82:0/100929493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee80b3bc0 msgr2=0x7faee80b4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.233+0000 7faeece00640 1 --2- 172.21.15.82:0/100929493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee80b3bc0 0x7faee80b4030 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faee0009d90 tx=0x7faee002bea0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.233+0000 7faeece00640 1 -- 172.21.15.82:0/100929493 shutdown_connections 2024-03-16T18:04:21.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.233+0000 7faeece00640 1 --2- 172.21.15.82:0/100929493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee80b3bc0 0x7faee80b4030 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.233+0000 7faeece00640 1 --2- 172.21.15.82:0/100929493 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faee806e310 0x7faee80cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.233+0000 7faeece00640 1 -- 172.21.15.82:0/100929493 >> 172.21.15.82:0/100929493 conn(0x7faee806ae70 msgr2=0x7faee806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:04:21.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.233+0000 7faeece00640 1 -- 172.21.15.82:0/100929493 shutdown_connections 2024-03-16T18:04:21.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.233+0000 7faeece00640 1 -- 172.21.15.82:0/100929493 wait complete. 2024-03-16T18:04:21.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.235+0000 7faeece00640 1 Processor -- start 2024-03-16T18:04:21.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.235+0000 7faeece00640 1 -- start start 2024-03-16T18:04:21.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.236+0000 7faeece00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee806e310 0x7faee81e6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.236+0000 7faeece00640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faee80b3bc0 0x7faee8133720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.236+0000 7faeece00640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faee81443c0 0x7faee8133c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.236+0000 7faeece00640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee812e840 con 0x7faee806e310 2024-03-16T18:04:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.236+0000 7faeece00640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee812e210 con 0x7faee81443c0 2024-03-16T18:04:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.236+0000 7faeece00640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee815c760 con 0x7faee80b3bc0 2024-03-16T18:04:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.237+0000 7faee7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee806e310 0x7faee81e6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.237+0000 7faee7400640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee806e310 0x7faee81e6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34744/0 (socket says 172.21.15.82:34744) 2024-03-16T18:04:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.237+0000 7faee7400640 1 -- 172.21.15.82:0/642569511 learned_addr learned my addr 172.21.15.82:0/642569511 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:04:21.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.237+0000 7faee7400640 1 -- 172.21.15.82:0/642569511 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faee80b3bc0 msgr2=0x7faee8133720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.237+0000 7faee7400640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faee80b3bc0 0x7faee8133720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.237+0000 7faee7400640 1 -- 172.21.15.82:0/642569511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faee81443c0 msgr2=0x7faee8133c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.237+0000 7faee7400640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faee81443c0 0x7faee8133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.237+0000 7faee7400640 1 -- 172.21.15.82:0/642569511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee0009a70 con 0x7faee806e310 2024-03-16T18:04:21.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.237+0000 7faee7400640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee806e310 0x7faee81e6de0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faed800c3c0 tx=0x7faed800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:21.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.238+0000 7faedf400640 1 -- 172.21.15.82:0/642569511 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8005ce0 con 0x7faee806e310 2024-03-16T18:04:21.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.238+0000 7faedf400640 1 -- 172.21.15.82:0/642569511 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed8005e40 con 0x7faee806e310 2024-03-16T18:04:21.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.238+0000 7faedf400640 1 -- 172.21.15.82:0/642569511 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8007310 con 0x7faee806e310 2024-03-16T18:04:21.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.238+0000 7faeece00640 1 -- 172.21.15.82:0/642569511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee815ca40 con 0x7faee806e310 2024-03-16T18:04:21.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.238+0000 7faeece00640 1 -- 172.21.15.82:0/642569511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee8134570 con 0x7faee806e310 2024-03-16T18:04:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.239+0000 7faeece00640 1 -- 172.21.15.82:0/642569511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee812e980 con 0x7faee806e310 2024-03-16T18:04:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.240+0000 7faedf400640 1 -- 172.21.15.82:0/642569511 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7faed8007530 con 0x7faee806e310 2024-03-16T18:04:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.241+0000 7faedf400640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faec806b3b0 0x7faec806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.241+0000 7faedf400640 1 -- 172.21.15.82:0/642569511 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faed8088e70 con 0x7faee806e310 2024-03-16T18:04:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.241+0000 7faee6a00640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faec806b3b0 0x7faec806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.242+0000 7faee6a00640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faec806b3b0 0x7faec806d870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faee8141740 tx=0x7faed4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:21.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.246+0000 7faedf400640 1 -- 172.21.15.82:0/642569511 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faed8053dc0 con 0x7faee806e310 2024-03-16T18:04:21.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.441+0000 7faeece00640 1 -- 172.21.15.82:0/642569511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7faee804f250 con 0x7faee806e310 2024-03-16T18:04:21.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.447+0000 7faedf400640 1 -- 172.21.15.82:0/642569511 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7faed80170c0 con 0x7faee806e310 2024-03-16T18:04:21.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.449+0000 7faedf400640 1 -- 172.21.15.82:0/642569511 <== mon.0 v2:172.21.15.82:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7faed8000ed0 con 0x7faee806e310 2024-03-16T18:04:21.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.451+0000 7faeece00640 1 -- 172.21.15.82:0/642569511 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faec806b3b0 msgr2=0x7faec806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.451+0000 7faeece00640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faec806b3b0 0x7faec806d870 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faee8141740 tx=0x7faed4008040 comp rx=0 tx=0).stop 2024-03-16T18:04:21.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.451+0000 7faeece00640 1 -- 172.21.15.82:0/642569511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee806e310 msgr2=0x7faee81e6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.451+0000 7faeece00640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee806e310 0x7faee81e6de0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7faed800c3c0 tx=0x7faed800c880 comp rx=0 tx=0).stop 2024-03-16T18:04:21.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.451+0000 7faeece00640 1 -- 172.21.15.82:0/642569511 shutdown_connections 2024-03-16T18:04:21.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.451+0000 7faeece00640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7faec806b3b0 0x7faec806d870 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.451+0000 7faeece00640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faee81443c0 0x7faee8133c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.451+0000 7faeece00640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7faee80b3bc0 0x7faee8133720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.451+0000 7faeece00640 1 --2- 172.21.15.82:0/642569511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee806e310 0x7faee81e6de0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.451+0000 7faeece00640 1 -- 172.21.15.82:0/642569511 >> 172.21.15.82:0/642569511 conn(0x7faee806ae70 msgr2=0x7faee806d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:04:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.452+0000 7faeece00640 1 -- 172.21.15.82:0/642569511 shutdown_connections 2024-03-16T18:04:21.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.452+0000 7faeece00640 1 -- 172.21.15.82:0/642569511 wait complete. 2024-03-16T18:04:21.476 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T18:04:21.476 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T18:04:21.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.675+0000 7fdac2800640 1 Processor -- start 2024-03-16T18:04:21.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.675+0000 7fdac2800640 1 -- start start 2024-03-16T18:04:21.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.676+0000 7fdac2800640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdabc1440a0 0x7fdabc06db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.676+0000 7fdac2800640 1 --2- >> v2:172.21.15.90:3300/0 conn(0x7fdabc144480 0x7fdabc06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.677+0000 7fdac2800640 1 -- --> v1:172.21.15.90:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc06e5c0 con 0x7fdabc15e430 2024-03-16T18:04:21.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.677+0000 7fdac2800640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc15efb0 con 0x7fdabc1440a0 2024-03-16T18:04:21.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.677+0000 7fdac2800640 1 -- --> v2:172.21.15.90:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc0ce670 con 0x7fdabc144480 2024-03-16T18:04:21.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.677+0000 7fdac1400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdabc1440a0 0x7fdabc06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:21.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.677+0000 7fdac1400640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdabc1440a0 0x7fdabc06db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34748/0 (socket says 172.21.15.82:34748) 2024-03-16T18:04:21.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.677+0000 7fdac1400640 1 -- 172.21.15.82:0/4221943748 learned_addr learned my addr 172.21.15.82:0/4221943748 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:04:21.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.677+0000 7fdac0a00640 1 --2- 172.21.15.82:0/4221943748 >> v2:172.21.15.90:3300/0 conn(0x7fdabc144480 0x7fdabc06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:21.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.677+0000 7fdac1400640 1 -- 172.21.15.82:0/4221943748 >> v2:172.21.15.90:3300/0 conn(0x7fdabc144480 msgr2=0x7fdabc06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:04:21.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.677+0000 7fdac1400640 1 --2- 172.21.15.82:0/4221943748 >> v2:172.21.15.90:3300/0 conn(0x7fdabc144480 0x7fdabc06e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.678+0000 7fdac1400640 1 -- 172.21.15.82:0/4221943748 >> v1:172.21.15.90:6789/0 conn(0x7fdabc15e430 legacy=0x7fdabc15e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T18:04:21.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.678+0000 7fdac1400640 1 -- 172.21.15.82:0/4221943748 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0ccff0 con 0x7fdabc1440a0 2024-03-16T18:04:21.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.679+0000 7fdac1400640 1 --2- 172.21.15.82:0/4221943748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1440a0 0x7fdabc06db40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdabc15e350 tx=0x7fdab002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bf3f621eb81e3e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:21.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.680+0000 7fdabbe00640 1 -- 172.21.15.82:0/4221943748 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0031870 con 0x7fdabc1440a0 2024-03-16T18:04:21.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.680+0000 7fdabbe00640 1 -- 172.21.15.82:0/4221943748 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdab00319d0 con 0x7fdabc1440a0 2024-03-16T18:04:21.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.680+0000 7fdabbe00640 1 -- 172.21.15.82:0/4221943748 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab0031d80 con 0x7fdabc1440a0 2024-03-16T18:04:21.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.680+0000 7fdac2800640 1 -- 172.21.15.82:0/4221943748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1440a0 msgr2=0x7fdabc06db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.680+0000 7fdac2800640 1 --2- 172.21.15.82:0/4221943748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1440a0 0x7fdabc06db40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fdabc15e350 tx=0x7fdab002b910 comp rx=0 tx=0).stop 2024-03-16T18:04:21.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.680+0000 7fdac2800640 1 -- 172.21.15.82:0/4221943748 shutdown_connections 2024-03-16T18:04:21.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.681+0000 7fdac2800640 1 --2- 172.21.15.82:0/4221943748 >> v2:172.21.15.90:3300/0 conn(0x7fdabc144480 0x7fdabc06e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.681+0000 7fdac2800640 1 --2- 172.21.15.82:0/4221943748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1440a0 0x7fdabc06db40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.681+0000 7fdac2800640 1 -- 172.21.15.82:0/4221943748 >> 172.21.15.82:0/4221943748 conn(0x7fdabc06ae70 msgr2=0x7fdabc06b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:04:21.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.681+0000 7fdac2800640 1 -- 172.21.15.82:0/4221943748 shutdown_connections 2024-03-16T18:04:21.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.681+0000 7fdac2800640 1 -- 172.21.15.82:0/4221943748 wait complete. 2024-03-16T18:04:21.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.683+0000 7fdac2800640 1 Processor -- start 2024-03-16T18:04:21.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.683+0000 7fdac2800640 1 -- start start 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.684+0000 7fdac2800640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdabc1440a0 0x7fdabc1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.684+0000 7fdac2800640 1 --2- >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdabc144480 0x7fdabc159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.684+0000 7fdac2800640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc15e430 0x7fdabc071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.684+0000 7fdac2800640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc159f60 con 0x7fdabc15e430 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.684+0000 7fdac2800640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0b3db0 con 0x7fdabc1440a0 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.684+0000 7fdac2800640 1 -- --> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0b3ad0 con 0x7fdabc144480 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.684+0000 7fdac1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc15e430 0x7fdabc071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.684+0000 7fdac1e00640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc15e430 0x7fdabc071eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34760/0 (socket says 172.21.15.82:34760) 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.684+0000 7fdac1e00640 1 -- 172.21.15.82:0/1948113447 learned_addr learned my addr 172.21.15.82:0/1948113447 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdac0a00640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdabc144480 0x7fdabc159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:21.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdac1e00640 1 -- 172.21.15.82:0/1948113447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdabc144480 msgr2=0x7fdabc159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdac1e00640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdabc144480 0x7fdabc159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdac1400640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdabc1440a0 0x7fdabc1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdac1e00640 1 -- 172.21.15.82:0/1948113447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdabc1440a0 msgr2=0x7fdabc1594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdac1e00640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdabc1440a0 0x7fdabc1594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdac1e00640 1 -- 172.21.15.82:0/1948113447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab00094d0 con 0x7fdabc15e430 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdac1e00640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc15e430 0x7fdabc071eb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdab400b990 tx=0x7fdab400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdab9600640 1 -- 172.21.15.82:0/1948113447 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab400cba0 con 0x7fdabc15e430 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdab9600640 1 -- 172.21.15.82:0/1948113447 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdab400cd00 con 0x7fdabc15e430 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdab9600640 1 -- 172.21.15.82:0/1948113447 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40146a0 con 0x7fdabc15e430 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.685+0000 7fdac2800640 1 -- 172.21.15.82:0/1948113447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc1411e0 con 0x7fdabc15e430 2024-03-16T18:04:21.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.686+0000 7fdac2800640 1 -- 172.21.15.82:0/1948113447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc1417a0 con 0x7fdabc15e430 2024-03-16T18:04:21.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.687+0000 7fdac2800640 1 -- 172.21.15.82:0/1948113447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc04f250 con 0x7fdabc15e430 2024-03-16T18:04:21.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.688+0000 7fdab9600640 1 -- 172.21.15.82:0/1948113447 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 89462+0+0 (secure 0 0 0) 0x7fdab4014800 con 0x7fdabc15e430 2024-03-16T18:04:21.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.689+0000 7fdab9600640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fda9c06b3b0 0x7fda9c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T18:04:21.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.689+0000 7fdab9600640 1 -- 172.21.15.82:0/1948113447 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdab4089970 con 0x7fdabc15e430 2024-03-16T18:04:21.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.689+0000 7fdac1400640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fda9c06b3b0 0x7fda9c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T18:04:21.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.690+0000 7fdac1400640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fda9c06b3b0 0x7fda9c06d870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdab0036000 tx=0x7fdab003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T18:04:21.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.695+0000 7fdab9600640 1 -- 172.21.15.82:0/1948113447 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdab4054830 con 0x7fdabc15e430 2024-03-16T18:04:21.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.933+0000 7fdac2800640 1 -- 172.21.15.82:0/1948113447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdabc0adc40 con 0x7fdabc15e430 2024-03-16T18:04:21.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.934+0000 7fdab9600640 1 -- 172.21.15.82:0/1948113447 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3693 (secure 0 0 0) 0x7fdab4019020 con 0x7fdabc15e430 2024-03-16T18:04:21.938 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-03-16T18:04:21.938 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/399724974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6833","nonce":399724974},{"type":"v1","addr":"172.21.15.90:6835","nonce":399724974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5100,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/3801030113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3801030113},{"type":"v1","addr":"172.21.15.82:6836","nonce":3801030113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5103,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2266371611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2266371611},{"type":"v1","addr":"172.21.15.82:6837","nonce":2266371611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5115,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6834/693422260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":693422260},{"type":"v1","addr":"172.21.15.90:6834","nonce":693422260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[]} 2024-03-16T18:04:21.938 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 18 2024-03-16T18:04:21.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.937+0000 7fdac2800640 1 -- 172.21.15.82:0/1948113447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fda9c06b3b0 msgr2=0x7fda9c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.937+0000 7fdac2800640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fda9c06b3b0 0x7fda9c06d870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fdab0036000 tx=0x7fdab003c040 comp rx=0 tx=0).stop 2024-03-16T18:04:21.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.937+0000 7fdac2800640 1 -- 172.21.15.82:0/1948113447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc15e430 msgr2=0x7fdabc071eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T18:04:21.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.937+0000 7fdac2800640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc15e430 0x7fdabc071eb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdab400b990 tx=0x7fdab400be50 comp rx=0 tx=0).stop 2024-03-16T18:04:21.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.937+0000 7fdac2800640 1 -- 172.21.15.82:0/1948113447 shutdown_connections 2024-03-16T18:04:21.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.937+0000 7fdac2800640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.82:6832/3774193927,v1:172.21.15.82:6833/3774193927] conn(0x7fda9c06b3b0 0x7fda9c06d870 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.937+0000 7fdac2800640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc15e430 0x7fdabc071eb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.937+0000 7fdac2800640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.90:3301/0,v1:172.21.15.90:6790/0] conn(0x7fdabc144480 0x7fdabc159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.937+0000 7fdac2800640 1 --2- 172.21.15.82:0/1948113447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdabc1440a0 0x7fdabc1594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T18:04:21.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.937+0000 7fdac2800640 1 -- 172.21.15.82:0/1948113447 >> 172.21.15.82:0/1948113447 conn(0x7fdabc06ae70 msgr2=0x7fdabc0cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T18:04:21.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.938+0000 7fdac2800640 1 -- 172.21.15.82:0/1948113447 shutdown_connections 2024-03-16T18:04:21.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-03-16T18:04:21.938+0000 7fdac2800640 1 -- 172.21.15.82:0/1948113447 wait complete. 2024-03-16T18:04:21.959 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-16T18:04:21.959 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-16T18:04:21.960 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-16T18:04:21.960 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:21.979 INFO:tasks.ceph.mds.a:Stopped 2024-03-16T18:04:21.979 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-16T18:04:21.979 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.048 INFO:tasks.ceph.mds.c:Stopped 2024-03-16T18:04:22.049 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-16T18:04:22.049 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.072 INFO:tasks.ceph.mds.b:Stopped 2024-03-16T18:04:22.072 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-16T18:04:22.073 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.135 INFO:tasks.ceph.mds.d:Stopped 2024-03-16T18:04:22.136 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-16T18:04:22.136 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-16T18:04:22.136 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.216 INFO:tasks.ceph.osd.0:Stopped 2024-03-16T18:04:22.217 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-16T18:04:22.217 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.290 INFO:tasks.ceph.osd.1:Stopped 2024-03-16T18:04:22.290 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-16T18:04:22.290 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.363 INFO:tasks.ceph.osd.2:Stopped 2024-03-16T18:04:22.363 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-16T18:04:22.363 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.443 INFO:tasks.ceph.osd.3:Stopped 2024-03-16T18:04:22.443 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-16T18:04:22.444 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.498 INFO:tasks.ceph.osd.4:Stopped 2024-03-16T18:04:22.499 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-16T18:04:22.499 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.563 INFO:tasks.ceph.osd.5:Stopped 2024-03-16T18:04:22.563 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-16T18:04:22.563 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.631 INFO:tasks.ceph.osd.6:Stopped 2024-03-16T18:04:22.631 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-16T18:04:22.631 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.692 INFO:tasks.ceph.osd.7:Stopped 2024-03-16T18:04:22.692 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-16T18:04:22.692 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-16T18:04:22.692 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.778 INFO:tasks.ceph.mgr.y:Stopped 2024-03-16T18:04:22.779 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-16T18:04:22.779 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.873 INFO:tasks.ceph.mgr.x:Stopped 2024-03-16T18:04:22.874 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-16T18:04:22.874 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-16T18:04:22.874 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.919 INFO:tasks.ceph.mon.a:Stopped 2024-03-16T18:04:22.920 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-16T18:04:22.920 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.954 INFO:tasks.ceph.mon.b:Stopped 2024-03-16T18:04:22.955 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-16T18:04:22.955 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T18:04:22.994 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-03-16T18:04:22.995 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-03-16T18:04:22.995 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-03-16T18:04:22.995 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-03-16T18:04:22.995 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-03-16T18:04:22.995 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-03-16T18:04:22.995 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-03-16T18:04:23.034 INFO:tasks.ceph.mon.c:Stopped 2024-03-16T18:04:23.034 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-16T18:04:23.034 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-03-16T18:04:23.142 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T18:04:23.142 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-16T18:04:23.406 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T18:04:23.406 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-16T18:04:23.559 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T18:04:23.559 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-16T18:04:23.719 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T18:04:23.720 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-16T18:04:23.877 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T18:04:23.877 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-16T18:04:24.198 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T18:04:24.198 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-16T18:04:24.332 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T18:04:24.332 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-16T18:04:24.475 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T18:04:24.475 DEBUG:teuthology.orchestra.run.smithi090:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-16T18:04:24.616 INFO:tasks.ceph:Archiving mon data... 2024-03-16T18:04:24.617 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024/data/mon.a.tgz 2024-03-16T18:04:24.617 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-03-16T18:04:24.675 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.s09GyHvZrV 2024-03-16T18:04:24.675 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.s09GyHvZrV 2024-03-16T18:04:24.857 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.s09GyHvZrV 2024-03-16T18:04:25.006 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.s09GyHvZrV is 325KB 2024-03-16T18:04:25.109 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.s09GyHvZrV 2024-03-16T18:04:25.164 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024/data/mon.b.tgz 2024-03-16T18:04:25.164 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-03-16T18:04:25.219 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.aRD4DaKA9F 2024-03-16T18:04:25.219 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.aRD4DaKA9F 2024-03-16T18:04:25.432 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.aRD4DaKA9F 2024-03-16T18:04:25.578 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.aRD4DaKA9F is 436KB 2024-03-16T18:04:25.690 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.aRD4DaKA9F 2024-03-16T18:04:25.746 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024/data/mon.c.tgz 2024-03-16T18:04:25.746 DEBUG:teuthology.orchestra.run.smithi090:> mktemp 2024-03-16T18:04:25.797 INFO:teuthology.orchestra.run.smithi090.stdout:/tmp/tmp.Lm5GhdcKI1 2024-03-16T18:04:25.797 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Lm5GhdcKI1 2024-03-16T18:04:25.987 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0666 /tmp/tmp.Lm5GhdcKI1 2024-03-16T18:04:26.140 DEBUG:teuthology.orchestra.remote:smithi090:/tmp/tmp.Lm5GhdcKI1 is 436KB 2024-03-16T18:04:26.249 DEBUG:teuthology.orchestra.run.smithi090:> rm -fr /tmp/tmp.Lm5GhdcKI1 2024-03-16T18:04:26.305 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-16T18:04:26.305 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-16T18:04:26.312 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-16T18:04:26.461 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-16T18:04:26.520 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-16T18:04:26.520 INFO:tasks.ceph:Archiving crash dumps... 2024-03-16T18:04:26.521 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024/remote/smithi082/crash 2024-03-16T18:04:26.522 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-16T18:04:26.616 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024/remote/smithi090/crash 2024-03-16T18:04:26.617 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-16T18:04:26.709 INFO:tasks.ceph:Shutting down logrotate 2024-03-16T18:04:26.710 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:04:26.717 DEBUG:teuthology.orchestra.run.smithi090:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T18:04:26.840 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-16T18:04:26.936 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-16T18:04:27.036 INFO:tasks.ceph:Compressing logs... 2024-03-16T18:04:27.036 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T18:04:27.044 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T18:04:27.125 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69335.log 2024-03-16T18:04:27.126 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43103.log 2024-03-16T18:04:27.127 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.69335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41848.log 2024-03-16T18:04:27.128 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69335.log.gz 2024-03-16T18:04:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37016.log 2024-03-16T18:04:27.128 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43103.log.gz 2024-03-16T18:04:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41848.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41848.log.gz 2024-03-16T18:04:27.130 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48586.log 2024-03-16T18:04:27.130 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-16T18:04:27.131 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48040.log 2024-03-16T18:04:27.131 INFO:teuthology.orchestra.run.smithi082.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.37016.log.gz 2024-03-16T18:04:27.133 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43441.log 2024-03-16T18:04:27.134 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48586.log.gz 2024-03-16T18:04:27.134 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48040.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48040.log.gz 2024-03-16T18:04:27.134 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44896.log 2024-03-16T18:04:27.135 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37371.log 2024-03-16T18:04:27.135 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37331.log 2024-03-16T18:04:27.135 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42842.log 2024-03-16T18:04:27.135 INFO:teuthology.orchestra.run.smithi090.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37371.log.gz 2024-03-16T18:04:27.135 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37331.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-16T18:04:27.135 INFO:teuthology.orchestra.run.smithi090.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37331.log.gz 2024-03-16T18:04:27.135 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.3.42842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42842.log.gz 2024-03-16T18:04:27.135 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-16T18:04:27.138 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42838.log 2024-03-16T18:04:27.138 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42451.log 2024-03-16T18:04:27.138 INFO:teuthology.orchestra.run.smithi090.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-16T18:04:27.139 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43441.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-16T18:04:27.139 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43441.log.gz 2024-03-16T18:04:27.139 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46061.log 2024-03-16T18:04:27.140 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44896.log.gz 2024-03-16T18:04:27.140 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48374.log 2024-03-16T18:04:27.140 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46061.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46061.log.gz 2024-03-16T18:04:27.140 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43764.log 2024-03-16T18:04:27.140 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48374.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.48374.log.gz 2024-03-16T18:04:27.140 INFO:teuthology.orchestra.run.smithi082.stderr: /var/log/ceph/ceph-client.admin.47834.log 2024-03-16T18:04:27.140 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48756.log 2024-03-16T18:04:27.144 INFO:teuthology.orchestra.run.smithi082.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.43764.log.gz 2024-03-16T18:04:27.144 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47834.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47834.log.gz 2024-03-16T18:04:27.144 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49008.log 2024-03-16T18:04:27.144 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107636.log 2024-03-16T18:04:27.144 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48756.log.gz 2024-03-16T18:04:27.144 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48458.log 2024-03-16T18:04:27.144 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49008.log.gz 2024-03-16T18:04:27.144 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105922.log 2024-03-16T18:04:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.3.42838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42838.log.gz 2024-03-16T18:04:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37171.log 2024-03-16T18:04:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.1.42451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42451.log.gz 2024-03-16T18:04:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-16T18:04:27.145 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37131.log 2024-03-16T18:04:27.146 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-osd.4.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37171.log.gz 2024-03-16T18:04:27.146 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37211.log 2024-03-16T18:04:27.146 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37131.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37131.log.gz 2024-03-16T18:04:27.146 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-16T18:04:27.146 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-16T18:04:27.148 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37211.log: /var/log/ceph/ceph-osd.7.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37211.log.gz 2024-03-16T18:04:27.148 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-16T18:04:27.148 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42432.log 2024-03-16T18:04:27.149 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107636.log.gz 2024-03-16T18:04:27.149 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105796.log 2024-03-16T18:04:27.149 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48458.log.gz 2024-03-16T18:04:27.149 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48248.log 2024-03-16T18:04:27.149 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105922.log.gz 2024-03-16T18:04:27.149 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107342.log 2024-03-16T18:04:27.150 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105796.log.gz 2024-03-16T18:04:27.150 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48966.log 2024-03-16T18:04:27.150 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48248.log.gz 2024-03-16T18:04:27.151 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37411.log 2024-03-16T18:04:27.151 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.1.42432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42432.log.gz 2024-03-16T18:04:27.151 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106058.log 2024-03-16T18:04:27.152 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107342.log.gz 2024-03-16T18:04:27.152 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48966.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48966.log.gz 2024-03-16T18:04:27.152 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48163.log 2024-03-16T18:04:27.153 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41904.log 2024-03-16T18:04:27.153 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106058.log.gz 2024-03-16T18:04:27.154 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-16T18:04:27.154 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37411.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37411.log.gz 2024-03-16T18:04:27.154 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37291.log 2024-03-16T18:04:27.157 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42294.log 2024-03-16T18:04:27.157 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48163.log.gz 2024-03-16T18:04:27.157 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45683.log 2024-03-16T18:04:27.157 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41904.log.gz 2024-03-16T18:04:27.157 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42294.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42294.log.gz 2024-03-16T18:04:27.157 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47918.log 2024-03-16T18:04:27.158 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-16T18:04:27.158 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-client.admin.37291.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37291.log.gz 2024-03-16T18:04:27.158 INFO:teuthology.orchestra.run.smithi090.stderr: 92.6%gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-16T18:04:27.158 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-03-16T18:04:27.160 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43350.log 2024-03-16T18:04:27.160 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45683.log.gz 2024-03-16T18:04:27.160 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107832.log 2024-03-16T18:04:27.160 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47918.log.gz 2024-03-16T18:04:27.160 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37275.log 2024-03-16T18:04:27.160 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43350.log.gz 2024-03-16T18:04:27.161 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mon.b.log: 93.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-16T18:04:27.161 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37251.log 2024-03-16T18:04:27.161 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107832.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107832.log.gz 2024-03-16T18:04:27.161 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44058.log 2024-03-16T18:04:27.162 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.37251.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37251.log.gz 2024-03-16T18:04:27.162 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- 93.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-16T18:04:27.163 INFO:teuthology.orchestra.run.smithi090.stderr: /var/log/ceph/ceph-mon.c.log 2024-03-16T18:04:27.164 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105122.log 2024-03-16T18:04:27.164 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37275.log.gz 2024-03-16T18:04:27.164 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47792.log 2024-03-16T18:04:27.164 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44058.log.gz 2024-03-16T18:04:27.165 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105122.log.gz 2024-03-16T18:04:27.165 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43094.log 2024-03-16T18:04:27.165 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105108.log 2024-03-16T18:04:27.166 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47792.log.gz 2024-03-16T18:04:27.167 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45767.log 2024-03-16T18:04:27.167 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43094.log.gz 2024-03-16T18:04:27.168 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105108.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105108.log.gz 2024-03-16T18:04:27.168 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106436.log 2024-03-16T18:04:27.169 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42770.log 2024-03-16T18:04:27.169 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45767.log.gz 2024-03-16T18:04:27.170 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45893.log 2024-03-16T18:04:27.170 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106436.log.gz 2024-03-16T18:04:27.171 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43440.log 2024-03-16T18:04:27.171 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42770.log.gz 2024-03-16T18:04:27.172 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47510.log 2024-03-16T18:04:27.172 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45893.log.gz 2024-03-16T18:04:27.173 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47384.log 2024-03-16T18:04:27.173 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43440.log.gz 2024-03-16T18:04:27.174 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41988.log 2024-03-16T18:04:27.175 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47510.log.gz 2024-03-16T18:04:27.175 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106310.log 2024-03-16T18:04:27.176 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47384.log.gz 2024-03-16T18:04:27.177 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41988.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41988.log.gz 2024-03-16T18:04:27.177 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45515.log 2024-03-16T18:04:27.177 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107216.log 2024-03-16T18:04:27.178 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106310.log.gz 2024-03-16T18:04:27.179 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106184.log 2024-03-16T18:04:27.179 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45515.log.gz 2024-03-16T18:04:27.180 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48416.log 2024-03-16T18:04:27.180 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107216.log.gz 2024-03-16T18:04:27.181 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106184.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106184.log.gz 2024-03-16T18:04:27.181 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46103.log 2024-03-16T18:04:27.182 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46466.log 2024-03-16T18:04:27.182 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48416.log.gz 2024-03-16T18:04:27.183 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107678.log 2024-03-16T18:04:27.183 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46103.log.gz 2024-03-16T18:04:27.184 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42764.log 2024-03-16T18:04:27.185 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46466.log.gz 2024-03-16T18:04:27.185 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107678.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107678.log.gz 2024-03-16T18:04:27.186 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45599.log 2024-03-16T18:04:27.186 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48206.log 2024-03-16T18:04:27.187 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42764.log.gz 2024-03-16T18:04:27.188 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106922.log 2024-03-16T18:04:27.188 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45599.log.gz 2024-03-16T18:04:27.189 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48206.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48206.log.gz 2024-03-16T18:04:27.189 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46754.log 2024-03-16T18:04:27.190 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47054.log 2024-03-16T18:04:27.191 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106922.log.gz 2024-03-16T18:04:27.191 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48542.log 2024-03-16T18:04:27.192 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46754.log.gz 2024-03-16T18:04:27.192 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43638.log 2024-03-16T18:04:27.193 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47054.log.gz 2024-03-16T18:04:27.193 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47960.log 2024-03-16T18:04:27.194 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48542.log.gz 2024-03-16T18:04:27.195 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43638.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43638.log.gz 2024-03-16T18:04:27.195 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45809.log 2024-03-16T18:04:27.196 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107174.log 2024-03-16T18:04:27.196 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47960.log.gz 2024-03-16T18:04:27.197 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49345.log 2024-03-16T18:04:27.197 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45809.log.gz 2024-03-16T18:04:27.198 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47468.log 2024-03-16T18:04:27.198 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107174.log.gz 2024-03-16T18:04:27.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.49345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49345.log.gz 2024-03-16T18:04:27.199 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42336.log 2024-03-16T18:04:27.200 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44184.log 2024-03-16T18:04:27.200 INFO:teuthology.orchestra.run.smithi082.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47468.log.gz 2024-03-16T18:04:27.201 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43401.log 2024-03-16T18:04:27.201 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42336.log.gz 2024-03-16T18:04:27.202 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47258.log 2024-03-16T18:04:27.202 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44184.log.gz 2024-03-16T18:04:27.203 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43401.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43401.log.gz 2024-03-16T18:04:27.203 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46970.log 2024-03-16T18:04:27.204 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42082.log 2024-03-16T18:04:27.204 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47258.log.gz 2024-03-16T18:04:27.205 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44268.log 2024-03-16T18:04:27.205 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46970.log.gz 2024-03-16T18:04:27.206 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45389.log 2024-03-16T18:04:27.207 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42082.log.gz 2024-03-16T18:04:27.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46670.log 2024-03-16T18:04:27.208 INFO:teuthology.orchestra.run.smithi082.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44268.log.gz 2024-03-16T18:04:27.209 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46424.log 2024-03-16T18:04:27.209 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45389.log.gz 2024-03-16T18:04:27.210 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47216.log 2024-03-16T18:04:27.210 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46670.log.gz 2024-03-16T18:04:27.211 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46424.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46424.log.gz 2024-03-16T18:04:27.211 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46271.log 2024-03-16T18:04:27.212 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43102.log 2024-03-16T18:04:27.212 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47216.log.gz 2024-03-16T18:04:27.213 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46796.log 2024-03-16T18:04:27.213 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46271.log.gz 2024-03-16T18:04:27.214 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43102.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-16T18:04:27.214 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43102.log.gz 2024-03-16T18:04:27.215 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45196.log 2024-03-16T18:04:27.215 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46796.log.gz 2024-03-16T18:04:27.216 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49092.log 2024-03-16T18:04:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.2.45196.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-16T18:04:27.217 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45196.log.gz 2024-03-16T18:04:27.217 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42546.log 2024-03-16T18:04:27.218 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44142.log 2024-03-16T18:04:27.221 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42546.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49092.log.gz 2024-03-16T18:04:27.223 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46928.log 2024-03-16T18:04:27.223 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42546.log.gz 2024-03-16T18:04:27.223 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43414.log 2024-03-16T18:04:27.223 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44142.log.gz 2024-03-16T18:04:27.223 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107132.log 2024-03-16T18:04:27.223 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46928.log.gz 2024-03-16T18:04:27.224 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48672.log 2024-03-16T18:04:27.224 INFO:teuthology.orchestra.run.smithi082.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43414.log.gz 2024-03-16T18:04:27.224 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49349.log 2024-03-16T18:04:27.225 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107132.log.gz 2024-03-16T18:04:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48672.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48672.log.gz 2024-03-16T18:04:27.225 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105880.log 2024-03-16T18:04:27.226 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.49349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49349.log.gz 2024-03-16T18:04:27.227 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46229.log 2024-03-16T18:04:27.228 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107048.log 2024-03-16T18:04:27.228 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105880.log.gz 2024-03-16T18:04:27.229 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46229.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30946.log 2024-03-16T18:04:27.229 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46229.log.gz 2024-03-16T18:04:27.230 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107594.log 2024-03-16T18:04:27.230 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107048.log.gz 2024-03-16T18:04:27.231 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.tmp-client.admin.30946.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30946.log.gz 2024-03-16T18:04:27.231 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45725.log 2024-03-16T18:04:27.232 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44100.log 2024-03-16T18:04:27.232 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107594.log.gz 2024-03-16T18:04:27.233 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43442.log 2024-03-16T18:04:27.233 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45725.log.gz 2024-03-16T18:04:27.234 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106604.log 2024-03-16T18:04:27.234 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44100.log.gz 2024-03-16T18:04:27.235 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43442.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43442.log.gz 2024-03-16T18:04:27.235 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106646.log 2024-03-16T18:04:27.236 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42210.log 2024-03-16T18:04:27.236 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106604.log.gz 2024-03-16T18:04:27.237 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43105.log 2024-03-16T18:04:27.237 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106646.log.gz 2024-03-16T18:04:27.238 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47300.log 2024-03-16T18:04:27.239 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42210.log.gz 2024-03-16T18:04:27.239 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43105.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43105.log.gz 2024-03-16T18:04:27.239 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107874.log 2024-03-16T18:04:27.240 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42769.log 2024-03-16T18:04:27.241 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47300.log.gz 2024-03-16T18:04:27.241 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47552.log 2024-03-16T18:04:27.242 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107874.log.gz 2024-03-16T18:04:27.242 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42769.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-16T18:04:27.243 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42769.log.gz 2024-03-16T18:04:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47552.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47552.log.gz 2024-03-16T18:04:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107510.log 2024-03-16T18:04:27.244 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42126.log 2024-03-16T18:04:27.245 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48714.log 2024-03-16T18:04:27.246 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107510.log.gz 2024-03-16T18:04:27.247 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47426.log 2024-03-16T18:04:27.247 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42126.log.gz 2024-03-16T18:04:27.248 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48290.log 89.0% 2024-03-16T18:04:27.248 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48714.log.gz 2024-03-16T18:04:27.249 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107468.log 2024-03-16T18:04:27.249 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47426.log.gz 2024-03-16T18:04:27.250 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48290.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48290.log.gz 2024-03-16T18:04:27.251 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46145.log 2024-03-16T18:04:27.252 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107468.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107468.log.gz 2024-03-16T18:04:27.252 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107792.log 2024-03-16T18:04:27.252 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46712.log 2024-03-16T18:04:27.253 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46145.log.gz 2024-03-16T18:04:27.254 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107792.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107792.log.gz 2024-03-16T18:04:27.254 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42462.log 2024-03-16T18:04:27.254 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105838.log 2024-03-16T18:04:27.255 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46712.log.gz 2024-03-16T18:04:27.256 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42462.log: gzip -5 --verbose 88.9% -- -- replaced with /var/log/ceph/ceph-client.admin.42462.log.gz /var/log/ceph/ceph-client.admin.107720.log 2024-03-16T18:04:27.256 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-16T18:04:27.257 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46508.log 2024-03-16T18:04:27.257 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105838.log.gz 2024-03-16T18:04:27.258 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107720.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107720.log.gz 2024-03-16T18:04:27.258 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106268.log 2024-03-16T18:04:27.259 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48332.log 2024-03-16T18:04:27.259 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46508.log.gz 2024-03-16T18:04:27.260 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106268.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106268.log.gz 2024-03-16T18:04:27.260 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107384.log 2024-03-16T18:04:27.261 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106478.log 2024-03-16T18:04:27.261 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48332.log.gz 2024-03-16T18:04:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107384.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107384.log.gz 2024-03-16T18:04:27.262 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43091.log 2024-03-16T18:04:27.263 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43106.log 2024-03-16T18:04:27.263 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106478.log.gz 2024-03-16T18:04:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43091.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43091.log.gz 2024-03-16T18:04:27.264 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-16T18:04:27.265 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45935.log 2024-03-16T18:04:27.265 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43106.log.gz 2024-03-16T18:04:27.266 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42378.log 2024-03-16T18:04:27.267 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45935.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45935.log.gz 2024-03-16T18:04:27.268 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47604.log 2024-03-16T18:04:27.268 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45641.log 2024-03-16T18:04:27.269 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42378.log.gz 2024-03-16T18:04:27.270 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107258.log 2024-03-16T18:04:27.270 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47604.log.gz 2024-03-16T18:04:27.271 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44767.log 2024-03-16T18:04:27.271 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45641.log.gz 2024-03-16T18:04:27.272 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46592.log 2024-03-16T18:04:27.272 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107258.log.gz 2024-03-16T18:04:27.273 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.44767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44767.log.gz 2024-03-16T18:04:27.274 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42759.log 2024-03-16T18:04:27.275 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46592.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46592.log.gz 2024-03-16T18:04:27.275 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41946.log 2024-03-16T18:04:27.276 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45977.log 2024-03-16T18:04:27.276 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42759.log.gz 2024-03-16T18:04:27.277 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107300.log 2024-03-16T18:04:27.278 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41946.log.gz 2024-03-16T18:04:27.279 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45347.log 2024-03-16T18:04:27.279 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45977.log.gz 2024-03-16T18:04:27.280 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49158.log 2024-03-16T18:04:27.280 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107300.log.gz 2024-03-16T18:04:27.281 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45851.log 2024-03-16T18:04:27.281 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45347.log.gz 2024-03-16T18:04:27.282 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106394.log 2024-03-16T18:04:27.282 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49158.log.gz 2024-03-16T18:04:27.283 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47749.log 2024-03-16T18:04:27.284 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45851.log.gz 2024-03-16T18:04:27.285 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47012.log 2024-03-16T18:04:27.285 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106394.log.gz 2024-03-16T18:04:27.286 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106964.log 2024-03-16T18:04:27.286 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47749.log.gz 2024-03-16T18:04:27.288 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47096.log 2024-03-16T18:04:27.288 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47012.log.gz 2024-03-16T18:04:27.289 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46550.log 2024-03-16T18:04:27.289 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106964.log.gz 2024-03-16T18:04:27.291 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42757.log 2024-03-16T18:04:27.291 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47096.log.gz 2024-03-16T18:04:27.292 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46187.log 2024-03-16T18:04:27.292 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46550.log.gz 2024-03-16T18:04:27.293 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37434.log 2024-03-16T18:04:27.293 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42757.log.gz 2024-03-16T18:04:27.294 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46187.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46187.log.gz -5 2024-03-16T18:04:27.294 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48121.log 2024-03-16T18:04:27.295 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37434.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-16T18:04:27.296 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37434.log.gz 2024-03-16T18:04:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2024-03-16T18:04:27.297 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48121.log.gz 2024-03-16T18:04:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45200.log 2024-03-16T18:04:27.299 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41579.log 2024-03-16T18:04:27.299 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2024-03-16T18:04:27.300 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.2.45200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48500.log 2024-03-16T18:04:27.300 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45200.log.gz 2024-03-16T18:04:27.301 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44406.log 2024-03-16T18:04:27.301 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41579.log.gz 2024-03-16T18:04:27.302 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107090.log 2024-03-16T18:04:27.302 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48500.log.gz 2024-03-16T18:04:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106142.log 2024-03-16T18:04:27.304 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44406.log.gz 2024-03-16T18:04:27.305 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41675.log 2024-03-16T18:04:27.305 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107090.log.gz 2024-03-16T18:04:27.305 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/ceph-mon.c.log: 92.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-16T18:04:27.306 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47342.log 2024-03-16T18:04:27.306 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106142.log.gz 2024-03-16T18:04:27.308 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43848.log 2024-03-16T18:04:27.308 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41675.log.gz 2024-03-16T18:04:27.309 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47342.log: gzip -5 --verbose -- /var/log/ceph/xfstests-dev-results/results/check.log 2024-03-16T18:04:27.309 INFO:teuthology.orchestra.run.smithi082.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.47342.log.gz 2024-03-16T18:04:27.310 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46838.log 2024-03-16T18:04:27.311 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43848.log.gz 2024-03-16T18:04:27.311 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/xfstests-dev-results/results/check.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43974.log 2024-03-16T18:04:27.312 INFO:teuthology.orchestra.run.smithi082.stderr: -3.0% -- replaced with /var/log/ceph/xfstests-dev-results/results/check.log.gz 2024-03-16T18:04:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45473.log 2024-03-16T18:04:27.314 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46838.log.gz 2024-03-16T18:04:27.315 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41756.log 2024-03-16T18:04:27.315 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43974.log.gz 2024-03-16T18:04:27.315 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-03-16T18:04:27.315 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45473.log.gz 2024-03-16T18:04:27.317 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42038.log 2024-03-16T18:04:27.317 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41756.log.gz 2024-03-16T18:04:27.318 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45431.log 2024-03-16T18:04:27.318 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-03-16T18:04:27.319 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47658.log 2024-03-16T18:04:27.319 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42038.log.gz 2024-03-16T18:04:27.320 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48924.log 2024-03-16T18:04:27.321 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45431.log.gz 2024-03-16T18:04:27.322 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47658.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-16T18:04:27.322 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47658.log.gz 2024-03-16T18:04:27.323 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106562.log 2024-03-16T18:04:27.323 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48924.log.gz 2024-03-16T18:04:27.324 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107006.log 2024-03-16T18:04:27.325 INFO:teuthology.orchestra.run.smithi090.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-16T18:04:27.326 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48798.log 2024-03-16T18:04:27.326 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106562.log.gz 2024-03-16T18:04:27.327 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107006.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107006.log.gz 2024-03-16T18:04:27.327 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42588.log 2024-03-16T18:04:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48798.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48798.log.gz 2024-03-16T18:04:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47174.log 2024-03-16T18:04:27.330 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69833.log 2024-03-16T18:04:27.330 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42588.log.gz 2024-03-16T18:04:27.331 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48882.log 2024-03-16T18:04:27.331 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47174.log.gz 2024-03-16T18:04:27.332 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.69833.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69833.log.gz 2024-03-16T18:04:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43404.log 2024-03-16T18:04:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48882.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44748.log 2024-03-16T18:04:27.334 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48882.log.gz 2024-03-16T18:04:27.335 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42252.log 2024-03-16T18:04:27.335 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43404.log.gz 2024-03-16T18:04:27.336 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.44748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44748.log.gz 2024-03-16T18:04:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-16T18:04:27.338 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42252.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42252.log.gz 2024-03-16T18:04:27.338 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42753.log 2024-03-16T18:04:27.340 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-16T18:04:27.340 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42753.log: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-16T18:04:27.341 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42753.log.gz 2024-03-16T18:04:27.342 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.47876.log 2024-03-16T18:04:27.343 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46313.log 2024-03-16T18:04:27.344 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47876.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47876.log.gz 2024-03-16T18:04:27.345 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-16T18:04:27.346 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46313.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46313.log.gz 2024-03-16T18:04:27.349 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43096.log 2024-03-16T18:04:27.350 INFO:teuthology.orchestra.run.smithi082.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-16T18:04:27.350 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49050.log 2024-03-16T18:04:27.352 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43439.log 2024-03-16T18:04:27.353 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49050.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43096.log.gz 2024-03-16T18:04:27.353 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49050.log.gz 2024-03-16T18:04:27.354 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106520.log 2024-03-16T18:04:27.355 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43439.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43439.log.gz 2024-03-16T18:04:27.356 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106226.log 2024-03-16T18:04:27.357 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106520.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106520.log.gz 2024-03-16T18:04:27.358 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106226.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106226.log.gz 2024-03-16T18:04:27.359 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42765.log 2024-03-16T18:04:27.361 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43099.log 2024-03-16T18:04:27.362 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42765.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42765.log.gz 2024-03-16T18:04:27.363 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43099.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43099.log.gz 2024-03-16T18:04:27.364 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48630.log 2024-03-16T18:04:27.365 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48840.log 2024-03-16T18:04:27.367 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48630.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48630.log.gz 2024-03-16T18:04:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48840.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48840.log.gz 2024-03-16T18:04:27.368 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42504.log 2024-03-16T18:04:27.370 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-03-16T18:04:27.371 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42504.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42504.log.gz 2024-03-16T18:04:27.372 INFO:teuthology.orchestra.run.smithi090.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-16T18:04:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47707.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-03-16T18:04:27.373 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45557.log 2024-03-16T18:04:27.375 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44226.log 2024-03-16T18:04:27.376 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45557.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45557.log.gz 2024-03-16T18:04:27.377 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44226.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44226.log.gz 2024-03-16T18:04:27.378 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106100.log 2024-03-16T18:04:27.380 INFO:teuthology.orchestra.run.smithi082.stderr:gzip 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-16T18:04:27.380 INFO:teuthology.orchestra.run.smithi082.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42168.log 2024-03-16T18:04:27.381 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106352.log 2024-03-16T18:04:27.381 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106100.log.gz 2024-03-16T18:04:27.382 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107552.log 2024-03-16T18:04:27.382 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42168.log.gz 2024-03-16T18:04:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106352.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106352.log.gz 2024-03-16T18:04:27.383 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43680.log 2024-03-16T18:04:27.384 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42767.log 2024-03-16T18:04:27.385 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8%/var/log/ceph/ceph-client.admin.43680.log: -- replaced with /var/log/ceph/ceph-client.admin.107552.log.gz 2024-03-16T18:04:27.386 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43680.log.gz 2024-03-16T18:04:27.386 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106688.log 2024-03-16T18:04:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42767.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42767.log.gz 2024-03-16T18:04:27.387 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46019.log 2024-03-16T18:04:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106688.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106688.log.gz 2024-03-16T18:04:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42420.log 2024-03-16T18:04:27.389 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43932.log 2024-03-16T18:04:27.391 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46019.log.gz 2024-03-16T18:04:27.391 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42420.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42420.log.gz 2024-03-16T18:04:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107426.log 2024-03-16T18:04:27.392 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43932.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43932.log.gz 2024-03-16T18:04:27.393 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-16T18:04:27.394 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43890.log 2024-03-16T18:04:27.395 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.107426.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107426.log.gz 2024-03-16T18:04:27.395 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43722.log 2024-03-16T18:04:27.397 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43890.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43890.log.gz 2024-03-16T18:04:27.397 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43722.log: 93.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-16T18:04:27.398 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105964.log 2024-03-16T18:04:27.398 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43722.log.gz 2024-03-16T18:04:27.399 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106880.log 2024-03-16T18:04:27.400 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.105964.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105037.log 2024-03-16T18:04:27.400 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105964.log.gz 2024-03-16T18:04:27.401 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.106880.log: /var/log/ceph/ceph-client.admin.105037.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106880.log.gz 2024-03-16T18:04:27.402 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.105037.log.gz 2024-03-16T18:04:27.439 INFO:teuthology.orchestra.run.smithi090.stderr: 93.3% 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-16T18:04:27.439 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-03-16T18:04:27.441 INFO:teuthology.orchestra.run.smithi090.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-16T18:04:27.459 INFO:teuthology.orchestra.run.smithi082.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-16T18:04:27.491 INFO:teuthology.orchestra.run.smithi090.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-16T18:04:27.496 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-03-16T18:04:27.496 INFO:teuthology.orchestra.run.smithi090.stderr:real 0m0.404s 2024-03-16T18:04:27.496 INFO:teuthology.orchestra.run.smithi090.stderr:user 0m1.695s 2024-03-16T18:04:27.496 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m0.249s 2024-03-16T18:04:27.536 INFO:teuthology.orchestra.run.smithi082.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-16T18:04:27.559 INFO:teuthology.orchestra.run.smithi082.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-16T18:04:27.564 INFO:teuthology.orchestra.run.smithi082.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-16T18:04:27.583 INFO:teuthology.orchestra.run.smithi082.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-16T18:04:27.586 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-03-16T18:04:27.587 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m0.500s 2024-03-16T18:04:27.587 INFO:teuthology.orchestra.run.smithi082.stderr:user 0m1.465s 2024-03-16T18:04:27.587 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m0.720s 2024-03-16T18:04:27.588 INFO:tasks.ceph:Archiving logs... 2024-03-16T18:04:27.588 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024/remote/smithi082/log 2024-03-16T18:04:27.589 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T18:04:27.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-03-16T18:04:27.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-03-16T18:04:27.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-03-16T18:04:27.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-03-16T18:04:27.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-03-16T18:04:27.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-03-16T18:04:27.996 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-03-16T18:04:28.010 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024/remote/smithi090/log 2024-03-16T18:04:28.011 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T18:04:28.310 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-16T18:04:28.322 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-16T18:04:28.322 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T18:04:28.329 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T18:04:28.482 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-16T18:04:28.493 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-16T18:04:28.493 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T18:04:28.501 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T18:04:28.546 INFO:teuthology.orchestra.run.smithi082.stderr:bash: line 1: ntpq: command not found 2024-03-16T18:04:28.555 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-03-16T18:04:28.557 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T18:04:28.557 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2024-03-16T18:04:28.557 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 44 -1247us[-1247us] +/- 92ms 2024-03-16T18:04:28.557 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -2313us[-2236us] +/- 50ms 2024-03-16T18:04:28.557 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +3952us[+4030us] +/- 63ms 2024-03-16T18:04:28.557 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T18:04:28.565 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-16T18:04:28.565 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-03-16T18:04:28.565 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 45 -1258us[-1063us] +/- 92ms 2024-03-16T18:04:28.566 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -2317us[-2123us] +/- 50ms 2024-03-16T18:04:28.566 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +3930us[+4124us] +/- 63ms 2024-03-16T18:04:28.566 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-16T18:04:28.568 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-16T18:04:28.579 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-16T18:04:28.580 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-16T18:04:28.598 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T18:04:28.821 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-16T18:04:29.032 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-16T18:04:29.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T18:04:29.134 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-16T18:04:29.230 INFO:teuthology.orchestra.run.smithi090.stdout:type=AVC msg=audit(1710611277.584:197): avc: denied { checkpoint_restore } for pid=1170 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1 2024-03-16T18:04:29.232 DEBUG:teuthology.task.selinux:ubuntu@smithi090.front.sepia.ceph.com has 1 denials 2024-03-16T18:04:29.232 ERROR:teuthology.run_tasks:Manager failed: selinux Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/__init__.py", line 136, in __exit__ self.teardown() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/selinux.py", line 166, in teardown self.get_new_denials() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/selinux.py", line 215, in get_new_denials raise SELinuxError(node=remote, teuthology.exceptions.SELinuxError: SELinux denials found on ubuntu@smithi090.front.sepia.ceph.com: ['type=AVC msg=audit(1710611277.584:197): avc: denied { checkpoint_restore } for pid=1170 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-03-16T18:04:29.233 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-16T18:04:29.244 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-16T18:04:29.258 INFO:teuthology.task.internal:Duration was 993.032471 seconds 2024-03-16T18:04:29.259 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-16T18:04:29.273 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-16T18:04:29.273 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T18:04:29.281 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T18:04:29.436 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T18:04:29.448 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-16T18:04:29.979 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-16T18:04:29.980 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-03-16T18:04:29.980 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T18:04:30.043 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-03-16T18:04:30.043 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T18:04:30.118 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-16T18:04:30.119 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T18:04:30.127 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T18:04:30.217 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-16T18:04:30.217 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T18:04:30.222 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T18:04:30.403 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-16T18:04:30.414 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-16T18:04:30.414 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T18:04:30.510 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T18:04:30.608 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-16T18:04:30.618 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T18:04:30.626 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T18:04:30.729 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-03-16T18:04:30.734 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-03-16T18:04:30.787 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T18:04:30.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T18:04:30.836 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T18:04:30.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T18:04:30.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-16T18:04:30.901 INFO:teuthology.task.internal:Transferring archived files... 2024-03-16T18:04:30.902 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024/remote/smithi082 2024-03-16T18:04:30.902 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T18:04:31.007 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024/remote/smithi090 2024-03-16T18:04:31.007 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T18:04:31.115 INFO:teuthology.task.internal:Removing archive directory... 2024-03-16T18:04:31.115 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T18:04:31.123 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T18:04:31.177 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-16T18:04:31.188 INFO:teuthology.task.internal:Not uploading archives. 2024-03-16T18:04:31.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-16T18:04:31.202 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-16T18:04:31.202 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T18:04:31.210 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T18:04:31.260 INFO:teuthology.orchestra.run.smithi082.stdout: 394126 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 18:04 /home/ubuntu/cephtest 2024-03-16T18:04:31.264 INFO:teuthology.orchestra.run.smithi090.stdout: 524321 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 18:04 /home/ubuntu/cephtest 2024-03-16T18:04:31.267 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-16T18:04:31.278 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-16T18:04:31.293 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: SELinuxError: SELinux denials found on ubuntu@smithi090.front.sepia.ceph.com: ['type=AVC msg=audit(1710611277.584:197): avc: denied { checkpoint_restore } for pid=1170 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'] 2024-03-16T18:04:31.293 INFO:teuthology.nuke:Checking targets against current locks 2024-03-16T18:04:31.320 DEBUG:teuthology.nuke:shortname: smithi082 2024-03-16T18:04:31.320 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T18:04:31.343 DEBUG:teuthology.nuke:shortname: smithi090 2024-03-16T18:04:31.343 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T18:04:31.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 17:38:05.455455', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMWDTjOWyp/xznfLFtNdGdjRytqkO/3ebZ6djnkvUAL2aYk9ZxFLqqX4KHBa7qOMy3OriVHGpGWzJ619MA+ZWK4='} 2024-03-16T18:04:31.367 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 17:38:05.454244', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMHGJiu0M9WpvHJtGRtV6+Bjf+tQAbbC+q4LpxhEK8u3bCpPcuXOnJC7Kux/5hu6k9cb6uWa/Jm96mHFtJ48DbI='} 2024-03-16T18:04:31.397 INFO:teuthology.orchestra.console.smithi082:Power off 2024-03-16T18:04:31.398 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T18:04:31.421 DEBUG:teuthology.orchestra.console.smithi082:power off output: Chassis Power Control: Down/Off 2024-03-16T18:04:31.421 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T18:04:31.439 INFO:teuthology.orchestra.console.smithi090:Power off 2024-03-16T18:04:31.440 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T18:04:31.457 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-03-16T18:04:31.461 DEBUG:teuthology.orchestra.console.smithi090:power off output: Chassis Power Control: Down/Off 2024-03-16T18:04:31.461 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T18:04:31.483 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-03-16T18:04:35.458 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T18:04:35.484 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T18:04:35.568 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-03-16T18:04:35.594 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is on 2024-03-16T18:04:39.570 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T18:04:39.595 DEBUG:teuthology.orchestra.console.smithi090:pexpect command: ipmitool -H smithi090.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T18:04:39.680 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is on 2024-03-16T18:04:39.706 DEBUG:teuthology.orchestra.console.smithi090:check power output: Chassis Power is off 2024-03-16T18:04:39.807 INFO:teuthology.orchestra.console.smithi090:Power off completed 2024-03-16T18:04:43.682 DEBUG:teuthology.orchestra.console.smithi082:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T18:04:43.792 DEBUG:teuthology.orchestra.console.smithi082:check power output: Chassis Power is off 2024-03-16T18:04:43.892 INFO:teuthology.orchestra.console.smithi082:Power off completed 2024-03-16T18:04:43.998 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 3845b7ac715a3ff198582638d199a90e30e254da client.1-kernel-sha1: 3845b7ac715a3ff198582638d199a90e30e254da description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/acls} duration: 993.0324709415436 failure_reason: 'SELinux denials found on ubuntu@smithi090.front.sepia.ceph.com: [''type=AVC msg=audit(1710611277.584:197): avc: denied { checkpoint_restore } for pid=1170 comm="agetty" capability=40 scontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tcontext=system_u:system_r:getty_t:s0-s0:c0.c1023 tclass=capability2 permissive=1'']' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-03-16T18:04:43.998 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T18:04:44.090 INFO:teuthology.run:FAIL